Blog | GitLab
Tutorials, product information, expert insights, and more from GitLab to help DevSecOps teams build, test, and deploy secure software faster.
The Git project recently released Git 2.49.0. Let's look at a few notable highlights from this release, which includes contributions from GitLab's Git team and the wider Git community. What's covered: git-backfill(1) and the new path-walk API Introduction of zlib-ng Continued iteration on Meson Deprecation of .git/branches/ and .git/remotes/ Rust bindings for libgit New name-hashing algorithm Promisor remote capability Thin clone using --revision git-backfill(1) and the new path-walk API When you git-clone(1) a Git repository, you can pass it the --filter option. Using this option allows you to create a partial clone. In a partial clone the server only sends a subset of reachable objects according to the given object filter. For example, creating a clone with --filter=blob:none will not fetch any blobs (file contents) from the server and create a blobless clone. Blobless clones have all the reachable commits and trees, but no blobs. When you perform an operation like git-checkout(1), Git will download the missing blobs to complete that operation. For some operations, like git-blame(1), this might result in downloading objects one by one, which will slow down the command drastically. This performance degradation occurs because git-blame(1) must traverse the commit history to identify which specific blobs it needs, then request each missing blob from the server separately. In Git 2.49, a new subcommand git-backfill(1) is introduced, which can be used to download missing blobs in a blobless partial clone. Under the hood, the git-backfill(1) command leverages the new path-walk API, which is different from how Git generally iterates over commits. Rather than iterating over the commits one at a time and recursively visiting the trees and blobs associated with each commit, the path-walk API does traversal by path. For each path, it adds a list of associated tree objects to a stack. This stack is then processed in a depth-first order. So, instead of processing every object in commit 1 before moving to commit 2, it will process all versions of file A across all commits before moving to file B. This approach greatly improves performance in scenarios where grouping by path is essential. Let me demonstrate its use by making a blobless clone of gitlab-org/git: $ git clone --filter=blob:none --bare --no-tags git@gitlab.com:gitlab-org/git.git Cloning into bare repository 'git.git'... remote: Enumerating objects: 245904, done. remote: Counting objects: 100% (1736/1736), done. remote: Compressing objects: 100% (276/276), done. remote: Total 245904 (delta 1591), reused 1547 (delta 1459), pack-reused 244168 (from 1) Receiving objects: 100% (245904/245904), 59.35 MiB | 15.96 MiB/s, done. Resolving deltas: 100% (161482/161482), done. Above, we use --bare to ensure Git doesn't need to download any blobs to check out an initial branch. We can verify this clone does not contain any blobs: $ git cat-file --batch-all-objects --batch-check='%(objecttype)' | sort | uniq -c 83977 commit 161927 tree If you want to see the contents of a file in the repository, Git has to download it: $ git cat-file -p HEAD:README.md remote: Enumerating objects: 1, done. remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 1 (from 1) Receiving objects: 100% (1/1), 1.64 KiB | 1.64 MiB/s, done. [](https://github.com/git/git/actions?query=branch%3Amaster+event%3Apush) Git - fast, scalable, distributed revision control system ========================================================= Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals. [snip] As you can see above, Git first talks to the remote repository to download the blob before it can display it. When you would like to git-blame(1) that file, it needs to download a lot more: $ git blame HEAD README.md remote: Enumerating objects: 1, done. remote: Counting objects: 100% (1/1), done. remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) Receiving objects: 100% (1/1), 1.64 KiB | 1.64 MiB/s, done. remote: Enumerating objects: 1, done. remote: Counting objects: 100% (1/1), done. remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) Receiving objects: 100% (1/1), 1.64 KiB | 1.64 MiB/s, done. remote: Enumerating objects: 1, done. remote: Counting objects: 100% (1/1), done. remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) Receiving objects: 100% (1/1), 1.64 KiB | 1.64 MiB/s, done. remote: Enumerating objects: 1, done. [snip] df7375d772 README.md (Ævar Arnfjörð Bjarmason 2021-11-23 17:29:09 +0100 1) [](https://github.com/git/git/actions?query=branch%3Amaster+event%3Apush) 5f7864663b README.md (Johannes Schindelin 2019-01-29 06:19:32 -0800 2) 28513c4f56 README.md (Matthieu Moy 2016-02-25 09:37:29 +0100 3) Git - fast, scalable, distributed revision control system 28513c4f56 README.md (Matthieu Moy 2016-02-25 09:37:29 +0100 4) ========================================================= 556b6600b2 README (Nicolas Pitre 2007-01-17 13:04:39 -0500 5) 556b6600b2 README (Nicolas Pitre 2007-01-17 13:04:39 -0500 6) Git is a fast, scalable, distributed revision control system with an 556b6600b2 README (Nicolas Pitre 2007-01-17 13:04:39 -0500 7) unusually rich command set that provides both high-level operations 556b6600b2 README (Nicolas Pitre 2007-01-17 13:04:39 -0500 8) and full access to internals. 556b6600b2 README (Nicolas Pitre 2007-01-17 13:04:39 -0500 9) [snip] We've truncated the output, but as you can see, Git goes to the server for each revision of that file separately. That's really inefficient. With git-backfill(1) we can ask Git to download all blobs: $ git backfill remote: Enumerating objects: 50711, done. remote: Counting objects: 100% (15438/15438), done. remote: Compressing objects: 100% (708/708), done. remote: Total 50711 (delta 15154), reused 14730 (delta 14730), pack-reused 35273 (from 1) Receiving objects: 100% (50711/50711), 11.62 MiB | 12.28 MiB/s, done. Resolving deltas: 100% (49154/49154), done. remote: Enumerating objects: 50017, done. remote: Counting objects: 100% (10826/10826), done. remote: Compressing objects: 100% (634/634), done. remote: Total 50017 (delta 10580), reused 10192 (delta 10192), pack-reused 39191 (from 1) Receiving objects: 100% (50017/50017), 12.17 MiB | 12.33 MiB/s, done. Resolving deltas: 100% (48301/48301), done. remote: Enumerating objects: 47303, done. remote: Counting objects: 100% (7311/7311), done. remote: Compressing objects: 100% (618/618), done. remote: Total 47303 (delta 7021), reused 6693 (delta 6693), pack-reused 39992 (from 1) Receiving objects: 100% (47303/47303), 40.84 MiB | 15.26 MiB/s, done. Resolving deltas: 100% (43788/43788), done. This backfills all blobs, turning the blobless clone into a full clone: $ git cat-file --batch-all-objects --batch-check='%(objecttype)' | sort | uniq -c 148031 blob 83977 commit 161927 tree This project was led by Derrick Stolee and was merged with e565f37553. Introduction of zlib-ng All objects in the .git/ folder are compressed by Git using zlib. zlib is the reference implementation for the RFC 1950: ZLIB Compressed Data Format. Created in 1995, zlib has a long history and is incredibly portable, even supporting many systems that predate the Internet. Because of its wide support of architectures and compilers, it has limitations in what it is capable of. The fork zlib-ng was created to accommodate the limitations. zlib-ng aims to be optimized for modern systems. This fork drops support for legacy systems and instead brings in patches for Intel optimizations, some Cloudflare optimizations, and a couple other smaller patches. The zlib-ng library itself provides a compatibility layer for zlib. The compatibility later allows zlib-ng to be a drop-in replacement for zlib, but that layer is not available on all Linux distributions. In Git 2.49: A compatibility layer was added to the Git project. Build options were added to both to the Makefile and Meson Build file. These additions make it easier to benefit from the performance improvements of zlib-ng. In local benchmarks, we've seen a ~25% speedup when using zlib-ng instead of zlib. And we're in the process of rolling out these changes to GitLab.com, too. If you want to benefit from the gains of zlib-ng, first verify if Git on your machine is already using zlib-ng by running git version --build-options: $ git version --build-options git version 2.47.1 cpu: x86_64 no commit associated with this build sizeof-long: 8 sizeof-size_t: 8 shell-path: /bin/sh libcurl: 8.6.0 OpenSSL: OpenSSL 3.2.2 4 Jun 2024 zlib: 1.3.1.zlib-ng If the last line includes zlib-ng then your Git is already built using the faster zlib variant. If not, you can either: Ask the maintainer of the Git package you are using to include zlib-ng support. Build Git yourself from source. These changes were introduced by Patrick Steinhardt. Continued iteration on Meson In our article about the Git 2.48 release, we touched on the introduction of the Meson build system. Meson is a build automation tool used by the Git project that at some point might replace Autoconf, CMake, and maybe even Make. During this release cycle, work continued on using Meson, adding various missing features and stabilization fixes: Improved test coverage for CI was merged in 72f1ddfbc9. Bits and pieces to use Meson in contrib/ were merged in 2a1530a953. Assorted fixes and improvements to the build procedure based on meson were merged in ab09eddf60. Making Meson aware of building git-subtree(1) was merged in 3ddeb7f337. Learn Meson to generate HTML documentation pages was merged in 1b4e9a5f8b. All these efforts were carried out by Patrick Steinhardt. Deprecation of .git/branches/ and .git/remotes/ You are probably aware of the existence of the .git directory, and what is inside. But have you ever heard about the sub-directories .git/branches/ and .git/remotes/? As you might know, reference to branches are stored in .git/refs/heads/, so that's not what .git/branches/ is for, and what about .git/remotes/? Way back in 2005, .git/branches/ was introduced to store a shorthand name for a remote, and a few months later they were moved to .git/remotes/. In 2006, git-config(1) learned to store remotes. This has become the standard way to configure remotes and, in 2011, the directories .git/branches/ and .git/remotes/ were documented as being "legacy" and no longer used in modern repositories. In 2024, the document BreakingChanges was started to outline breaking changes for the next major version of Git (v3.0). While this release is not planned to happen any time soon, this document keeps track of changes that are expected to be part of that release. In 8ccc75c245, the use of the directories .git/branches/ and .git/remotes/ was added to this document and that officially marks as them deprecated and to be removed in Git 3.0. Thanks to Patrick Steinhardt for formalizing this deprecation. Rust bindings for libgit When compiling Git, an internal library libgit.a is made. This library contains some of the core functionality of Git. While this library (and most of Git) is written in C, in Git 2.49 bindings were added to make some of these functions available in Rust. To achieve this, two new Cargo packages were created: libgit-sys and libgit-rs. These packages live in the contrib/ subdirectory in the Git source tree. It's pretty common to split out a library into two packages when a Foreign Function Interface is used. The libgit-sys package provides the pure interface to C functions and links to the native libgit.a library. The package libgit-rs provides a high-level interface to the functions in libgit-sys with a feel that is more idiomatic to Rust. So far, the functionality in these Rust packages is very limited. It only provides an interface to interact with the git-config(1). This initiative was led by Josh Steadmon and was merged with a4af0b6288. New name-hashing algorithm The Git object database in .git/ stores most of its data in packfiles. And packfiles are also used to submit objects between Git server and client over the wire. You can read all about the format at gitformat-pack(5). One important aspect of the packfiles is delta-compression. With delta-compression not every object is stored as-is, but some objects are saved as a delta of another base. So instead of saving the full contents of the objects, changes compared to another object are stored. Without going into the details how these deltas are calculated or stored, you can imagine that it is important group files together that are very similar. In v2.48 and earlier, Git looked at the last 16 characters of the path name to determine whether blobs might be similar. This algorithm is named version 1. In Git 2.49, version 2 is available. This is an iteration on version 1, but modified so the effect of the parent directory is reduced. You can specify the name-hash algorithm version you want to use with option --name-hash-version of git-repack(1). Derrick Stolee, who drove this project, did some comparison in resulting packfile size after running git repack -adf --name-hash-version=: Repo Version 1 size Version 2 size fluentui 440 MB 161 MB Repo B 6,248 MB 856 MB Repo C 37,278 MB 6,921 MB Repo D 131,204 MB 7,463 MB You can read more of the details in the patch set, which is merged in aae91a86fb. Promisor remote capability It's known that Git isn't great in dealing with large files. There are some solutions to this problem, like Git LFS, but there are still some shortcomings. To give a few: With Git LFS the user has to configure which files to put in LFS. The server has no control about that and has to serve all files. Whenever a file is committed to the repository, there is no way to get it out again without rewriting history. This is annoying, especially for large files, because they are stuck for eternity. Users cannot change their mind on which files to put into Git LFS. A tool like Git LFS requires significant effort to set up, learn, and use correctly. For some time, Git has had the concept of promisor remotes. This feature can be used to deal with large files, and in Git 2.49 this feature took a step forward. The idea for the new “promisor-remote” capability is relatively simple: Instead of sending all objects itself, a Git server can tell to the Git client "Hey, go download these objects from XYZ". XYZ would be a promisor remote. Git 2.49 enables the server to advertise the information of the promisor remote to the client. This change is an extension to gitprotocol-v2. While the server and the client are transmitting data to each other, the server can send names and URLs of the promisor remotes it knows about. So far, the client is not using the promisor remote info it gets from the server during clone, so all objects are still transmitted from the remote the clone initiated from. We are planning to continue work on this feature, making it use promisor remote info from the server, and making it easier to use. This patch set was submitted by Christian Couder and merged with 2c6fd30198. Thin clone using --revision A new --revision option was added to git-clone(1). This enables you to create a thin clone of a repository that only contains the history of the given revision. The option is similar to --branch, but accepts a ref name (like refs/heads/main, refs/tags/v1.0, and refs/merge-requests/123) or a hexadecimal commit object ID. The difference to --branch is that it does not create a tracking branch and detaches HEAD. This means it's not suited if you want to contribute back to that branch. You can use --revision in combination with --depth to create a very minimal clone. A suggested use-case is for automated testing. When you have a CI system that needs to check out a branch (or any reference) to perform autonomous testing on the source code, having a minimal clone is all you need. This change was driven by Toon Claes. Read more What’s new in Git 2.48.0? What’s new in Git 2.47.0? What’s new in Git 2.46.0?
Welcome to our "Getting started with GitLab" series, where we help newcomers get familiar with the GitLab DevSecOps platform. This post dives into the gitlab-triage gem, a powerful tool that lets you create bots to automate your Agile workflow. Say goodbye to manual tasks and hello to streamlined efficiency. Why automate your workflow? Efficiency is key in software development. Automating repetitive tasks like triaging issues and merge requests frees up valuable time for your team to focus on what matters most: building amazing software. With gitlab-triage, you can: Ensure consistency: Apply labels and assign issues automatically based on predefined rules. Improve response times: Get immediate feedback on new issues and merge requests. Reduce manual effort: Eliminate the need for manual triage and updates. Boost productivity: Free up your team to focus on coding and innovation. Introducing the gitlab-triage gem The gitlab-triage gem is a Ruby library that allows you to create bots that interact with your GitLab projects. These bots can automatically perform a wide range of actions, including: Labeling: Automatically categorize issues and merge requests. Commenting: Provide updates, request information, or give feedback. Assigning: Assign issues and merge requests to the appropriate team members. Closing: Close stale or resolved issues and merge requests. Creating: Generate new issues based on specific events or conditions. And much more! Check out the gitlab-triage gem repository. Setting up your triage bot Let's get your first triage bot up and running! Install the gem. (Note: The gem command is available with Ruby programming language installed.) gem install gitlab-triage Get your GitLab API token. Go to your GitLab profile settings. Navigate to Access Tokens. Create a new token with the api scope. Keep your token secure and set an expiration date for it based on when you will be done with this walkthrough! Define your triage policies. Create a file named .triage-policies.yml in your project's root directory. This file will contain the rules that govern your bot's behavior. Here's a simple example: --- - name: "Apply 'WIP' label" condition: draft: true action: labels: - status::wip - name: "Request more information on old issue" condition: date: attribute: updated_at condition: older_than interval_type: months interval: 12 action: comment: | {{author}} This issue has been open for more than 12 months, is this still an issue? This configuration defines two policies: The first policy applies the status::wip label to any issue that is in draft. The second policy adds a comment to an issue that the issue has not been updated in 12 months. Run your bot. You can run your bot manually using the following command: gitlab-triage -t -p Replace with your GitLab API token and with the ID of your GitLab project. If you would like to see the impact of actions before they are taken, you can add the -n or --dry-run to test out the policies first. Automating with GitLab CI/CD To automate the execution of your triage bot, integrate it with GitLab CI/CD. Here's an example .gitlab-ci.yml configuration: triage: script: - gem install gitlab-triage - gitlab-triage -t $GITLAB_TOKEN -p $CI_PROJECT_ID only: - schedules This configuration defines a job named "triage" that installs the gitlab-triage gem and runs the bot using the $GITLAB_TOKEN (a predefined CI/CD variable) and the $CI_PROJECT_ID variable. The only: schedules clause ensures that the job runs only on a schedule. To create a schedule, go to your project's CI/CD settings and navigate to Schedules. Create a new schedule and define the frequency at which you want your bot to run (e.g., daily, hourly). Advanced triage policies gitlab-triage offers a range of advanced features for creating more complex triage policies: Regular expressions: Use regular expressions for more powerful pattern matching. Summary policies: Consolidate related issues into a single summary issue. Custom actions: Define custom actions using Ruby code blocks to perform more complex operations using the GitLab API. Here are two advanced real-world examples from the triage bot used by the Developer Advocacy team at GitLab. You can view the full policies in this file. - name: Issues where DA team member is an assignee outside DA-Meta project i.e. DevRel-Influenced conditions: assignee_member: source: group condition: member_of source_id: 1008 state: opened ruby: get_project_id != 18 forbidden_labels: - developer-advocacy actions: labels: - developer-advocacy - DevRel-Influenced - DA-Bot::Skip This example for issues across a group, excluding those in the project with the ID of 18, have assignees who are members of the group with ID of 1008 and do not have the label developer-advocacy on them. This policy helps the Developer Advocacy team at GitLab to find issues members of the team are assigned to but are not in their team’s project. This helps the team identify and keep track of contributions made outside of the team by adding the teams’ labels. - name: Missing Due Dates conditions: ruby: missing_due_date state: opened labels: - developer-advocacy forbidden_labels: - DA-Due::N/A - DA-Bot::Skip - DA-Status::FYI - DA-Status::OnHold - CFP - DA-Bot::Triage actions: labels: - DA-Bot-Auto-Due-Date comment: | /due #{get_current_quarter_last_date} This second example checks for all issues with the developer-advocacy label, which do not include labels in the forbidden labels list and when their due dates have passed. It updates the due dates automatically by commenting on the issue with a slash command and a date that is generated using Ruby. The Ruby scripts used in the policies are defined in a separate file as shown below. This feature allows you to be flexible in working with your filters and actions. You can see functions are created for different Ruby commands that we used in our policies. require 'json' require 'date' require "faraday" require 'dotenv/load' module DATriagePlugin def last_comment_at conn = Faraday.new( url: notes_url+"?sort=desc&order_by=created_at&pagination=keyset&per_page=1", headers: {'PRIVATE-TOKEN' => ENV.fetch("PRIV_KEY"), 'Content-Type' => 'application/json' } ) response = conn.get() if response.status == 200 jsonData = JSON.parse(response.body) if jsonData.length > 0 Date.parse(jsonData[0]['created_at']) else Date.parse(resource[:created_at]) end else Date.parse(resource[:created_at]) end end def notes_url resource[:_links][:notes] end def get_project_id resource[:project_id] end def get_current_quarter_last_date() yr = Time.now.year case Time.now.month when 2..4 lm = 4 when 5..7 lm = 7 when 8..10 lm = 10 when 11..12 lm = 1 yr = yr + 1 else lm = 1 end return Date.new(yr, lm, -1) end def one_week_to_due_date if(resource[:due_date] == nil) false else days_to_due = (Date.parse(resource[:due_date]) - Date.today).to_i if(days_to_due > 0 && days_to_due Date.parse(resource[:due_date]) end end def missing_due_date if(resource[:due_date] == nil) true else false end end end Gitlab::Triage::Resource::Context.include DATriagePlugin The triage bot is executed using the command: `gitlab-triage -r ./triage_bot/issue_triage_plugin.rb --debug --token $PRIV_KEY --source-id gitlab-com --source groups` -r: Passes in a file of requirements for the performing triage. In this case we are passing in our Ruby functions. --debug: Prints debugging information as part of the output. --token: Is used to pass in a valid GitLab API token. --source: Specifies if the sources of the issues it will search is within a group or a project. --source-id: Takes in the ID of the selected source type – in this case, a group. The GitLab triage-ops project is another another real-world example that is more complex and you can learn how to build your own triage bot. Best practices Start simple: Begin with basic policies and gradually increase complexity as needed. Test thoroughly: Test your policies in a staging environment before deploying them to production. Monitor regularly: Monitor your bot's activity to ensure it's behaving as expected. Use descriptive names: Give your policies clear and descriptive names for easy maintenance. Be mindful of the scope of your filters: You might be tempted to filter issues across groups where thousands of issues exist. However, this can slow down the triage and also make the process fail due to rate limitations against the GitLab API. Prioritize using labels for triages: To avoid spamming other users, labels are a good way to perform triages without cluttering comments and issues. Take control of your workflow With the gitlab-triage gem, you can automate your GitLab workflow and unlock new levels of efficiency. Start by creating simple triage bots and gradually explore the more advanced features. You'll be amazed at how much time and effort you can save! Getting started with GitLab series Check out more articles in our "Getting Started with GitLab" series below: How to manage users How to import your projects to GitLab Mastering project management
Development and security teams are often overwhelmed by the number of vulnerabilities they need to remediate. Many organizations remediate less than 16% of their known vulnerabilities monthly. Vulnerability management teams face a constant challenge: which security flaws deserve immediate attention? Three key frameworks help answer this question: Common Vulnerability Scoring System (CVSS), Known Exploited Vulnerabilities (KEV), and Exploit Prediction Scoring System (EPSS). The GitLab 17.9 release adds support for these frameworks. In this article, you'll learn how to use these frameworks within GitLab to efficiently prioritize risk across your dependency and container image vulnerabilities using this data. Vulnerability risk assessment data CVSS CVSS provides a standardized method for rating the severity of security vulnerabilities. Scores range from 0 to 10, with higher values indicating greater severity. CVSS evaluates vulnerabilities across three dimension groups: Base metrics: intrinsic qualities that don't change over time (attack complexity, privileges required) Temporal metrics: factors that evolve (exploit maturity, remediation level) Environmental metrics: organization-specific impact considerations CVSS offers a consistent severity baseline and common language for security teams. Its comprehensive scoring methodology considers multiple aspects of a vulnerability's technical impact. KEV The Cybersecurity and Infrastructure Security Agency (CISA) maintains the KEV catalog, which identifies vulnerabilities actively exploited in the wild. Unlike academic severity scores, KEV focuses on real-world threat intelligence. Each entry includes: CVE identifier Vulnerability name Action required Due date for remediation (for federal agencies) KEV provides actionable intelligence based on observed threat actor behavior. It cuts through scoring complexity with a binary signal: "This vulnerability is being actively exploited right now." EPSS The EPSS uses machine learning to predict the likelihood a vulnerability will be exploited in the next 30 days. Scores range from 0 to 1 (or 0%-100%), representing probability. EPSS analyzes hundreds of factors, including: Technical characteristics Social media mentions Exploit availability Vulnerability age EPSS brings risk-based prioritization to vulnerability management. Rather than focusing solely on technical severity, it helps teams understand which vulnerabilities attackers are most likely to target. Combining the frameworks for effective prioritization Each framework serves a unique purpose: CVSS indicates how severe a vulnerability is technically. KEV indicates which vulnerabilities are actively being exploited. EPSS indicates which vulnerabilities are likely to be exploited soon. An effective prioritization strategy leverages all three: Start with KEV-listed vulnerabilities as immediate priorities. Use EPSS to identify high-probability threats not yet on KEV. Consider CVSS for understanding technical impact. By combining these complementary frameworks, security teams can focus limited resources on the vulnerabilities that pose the greatest actual risk to their organizations. You can get started with prioritizing vulnerabilities with GitLab by: Adding security scanners to your pipeline Viewing vulnerability insights Setting the vulnerability status based metrics Watch this video to learn more: Adding security scanners to your pipeline GitLab provides built-in security scanning tools through its templates that can be integrated directly into your CI/CD pipeline. GitLab offers several security scanners that address different aspects of your application security: Static Application Security Testing (SAST): Analyzes your source code for known vulnerabilities Dynamic Application Security Testing (DAST): Tests your running application for vulnerabilities Dependency Scanning: Checks your dependencies for known vulnerabilities Container Scanning: Identifies vulnerabilities in container images Secret Detection: Finds secrets and credentials accidentally committed to your repository Infrastructure as Code Scanning: Detects security issues in IaC files Coverage-guided Fuzzing: Sends random inputs to an instrumented version of your application in an effort to detect bugs Web API Fuzzing: Sets operation parameters to unexpected values in an effort to cause unexpected behavior and errors in the API backend To add them to your pipeline, simply add the appropriate templates to .gitlab-ci.yml file. For example, adding SAST and Dependency Scanning to your pipeline is as simple as: include: - template: Security/SAST.gitlab-ci.yml - template: Security/Dependency-Scanning.gitlab-ci.yml stages: - test Once you commit the above changes, security scanners will begin to run. These scanners can be further configured to meet the needs of your organization. To learn more about our various scanners, see the GitLab application security documentation. Note: EPSS and KEV metrics are only provided for dependency and container image vulnerabilities. Viewing vulnerability insights Once a pipeline with your security scanners is run on the default branch, you can access the vulnerability report. The vulnerability report provides a consolidated view of all security vulnerabilities detected across your project by GitLab's security scanners. You can access it from your project by going to the side-tab and selecting Secure > Vulnerability Report. Vulnerability report grouped by tool From the vulnerability report, select a vulnerability to see its insights page, which includes the severity, EPSS, KEV, and CVSS along with the following: description when it was detected current status available actions linked issues actions log filename and line number of the vulnerability (if available) This data can be used to effectively triage, remediate, or mitigate the vulnerability. Note: From the insights page, you can also leverage GitLab Duo’s AI capabilities to explain and auto-resolve a vulnerability. Setting the vulnerability status-based metrics After examining the provided data, we can go ahead and change the status of our vulnerability by clicking the Change status button: Change vulnerability status from insights page Then we'll see a popup that will allow you to change the status of a vulnerability: Change vulnerability status option When you dismiss a vulnerability you can choose one of the following reasons and optionally provide a comment: Acceptable risk: The vulnerability is known, and has not been remediated or mitigated, but is considered to be an acceptable business risk. False positive: An error in reporting in which a test result incorrectly indicates the presence of a vulnerability in a system when the vulnerability is not present. Mitigating control: The vulnerability’s risk is mitigated by a management, operational, or technical control (that is, safeguard or countermeasure) employed by an organization that provides equivalent or comparable protection for an information system. Used in tests: The finding is not a vulnerability because it is part of a test or is test data. Not applicable: The vulnerability is known, and has not been remediated or mitigated, but is considered to be in a part of the application that will not be updated. And there you have it, quick and easy vulnerability risk prioritization with GitLab! Get started today with a free, 60-day trial of GitLab Ultimate! Learn more To learn more about GitLab security and governance features and how we can help enhance your security posture, check out the following resources: GitLab Risk Assessment Data GitLab Security and Compliance Solutions GitLab Application Security documentation GitLab Risk Assessment Data epic
At GitLab, we’ve implemented an innovative approach to improving our experience called Beautifying our UI. This unique initiative pairs one product designer with a frontend engineer for a milestone or two, and empowers them to make self-directed usability improvements across the platform. Ultimately, this helps build a more polished product experience, as these pairs can quickly address pain points, refine interactions, and deliver thoughtful improvements that make the platform more efficient and enjoyable to use. In this iteration, Anna Vovchenko and I decided to focus on the continuous deployment (CD) area of the product. Here is how we did it and what we learned. Trying something new As this was our second round going through the process, we wanted to make several small adjustments that in the end helped us deliver even more quality improvements to the product. These process improvements included: Extended timeline: We decided this time around we wanted to extend the initiative to span two milestones. This gave us the time to tackle more complex problems, but also gave us space for additional planning at the start. Structured planning: While it was encouraged in the past to work directly in merge requests, we found it helped to use the initial issue as a place to plan and seek out problems ahead of time. Rather than purely focusing on the ad-hoc, we incorporated a planning phase similar to milestone planning, helping the partnership identify and prioritize potential improvements beforehand. Product manager integration: As we focused on one area for this round of the project, we also decided to involve the product manager of the team more actively in the process. This ensured alignment on larger changes, reduced surprises when MRs were merged and allowed us to gather valuable feedback throughout the implementation. Engaging the community: We expanded our improvement efforts by inviting contributions from community members, accelerating our ability to implement fixes and enhancements across the platform. Strategic timing: We chose to run this iteration during a traditionally slower period, allowing teams to focus more deeply on these improvements without competing priorities. These refinements maintained the initiative's core strength of direct designer-engineer collaboration, while adding structure that helped our pair work more effectively. What were the main improvements? During the two milestones, our pairing implemented several significant improvements that enhance the user experience across the CD space. Here's a look at what we accomplished: Enhanced environment list view One of the larger changes made during this cycle of "Beautifying our UI" was a redesigned Environment List page to make deployment information more accessible. Previously, users had to click through collapsible sections to view crucial deployment details, and viewing important details at a glance was difficult. Now, this information is immediately visible, bringing the most important deployment information to the forefront where users need it. Before: The original design relied on collapsible sections, requiring users to click to reveal deployment information. This meant that users couldn't immediately see the status of their deployments, making it harder to quickly assess the state of their environments. After: The new design surfaces critical deployment information directly in the list view, including: Deployment status with clear visual indicators Who triggered the deployment along with timestamps Commit information and version tags Actions to take on the environment Latest deployment indicators This redesign eliminates the need for extra clicks and gives users immediate visibility into their deployment and environment statuses. The new layout maintains a clean interface while presenting more actionable information upfront. Improved deploy keys filtering Another larger enhancement was made to our deploy keys interface to improve searchability while maintaining performance. This change addresses a critical user need for quickly finding specific deploy keys in large repositories, which was broken when pagination was introduced earlier last year. Before: The previous interface displayed deploy keys in a paginated list without a dedicated search function. While pagination helped with performance when handling thousands of keys, users had lost the ability to quickly search through their deploy keys using the browser search functionality, forcing them to manually scan through multiple pages. After: The new design introduces a dedicated search field at the top of the deploy keys list, allowing users to: Quickly filter deploy keys by name or SHA Maintain the performance benefits of pagination Find specific keys without browsing through multiple pages This improvement strikes the right balance between performance and usability, especially beneficial for teams managing numerous deploy keys across multiple projects. Better Kubernetes agent management We made significant improvements to the Kubernetes agent experience by simplifying the registration process and providing better visibility into agent status. These enhancements work together to create a smoother onboarding experience for teams getting started. Our first area of focus was streamlining how users register agents when they have configuration files ready to use. Previously, this process had several pain points that we wanted to address. Before: Only showed connected and previously connected agents Connection status was limited to "Never connected" or "Not connected" No clear path to register new agents After: Added a new Available configurations tab showing all potential agent configurations Clear "Register an agent" call-to-action button for each available configuration Next, we turned our attention to making the agent registration modal more intuitive. The previous design created some confusion that we wanted to resolve. Before: Users faced a confusing dual-purpose search box that both found existing agents and created new ones The workflow had too many decision points instead of a clear path forward The process for creating vs. selecting an agent wasn't clearly separated After: Separated the interface into two clear options: bootstrap with Flux or create an agent through the UI Streamlined the workflow into a more linear process Made the distinction between creating new agents and selecting existing ones more obvious Added a success message that clearly shows where to create the optional config file These improvements make it immediately clear which agents need attention and provide a straightforward path to register new agents. The reorganized interface better supports both new users setting up their first agent and experienced users managing multiple agents. Additional usability enhancements While working on major interface improvements, we also addressed several focused usability issues that significantly improve the day-to-day experience: Enhanced Kubernetes pod search: Added search functionality for Kubernetes pods on the environment page, making it easier to locate specific pods in large deployments. This was showcased in the GitLab 17.8 release post. Improved Flux status visibility: Added a "stopped" badge to the dashboard view when Flux sync is stopped, providing immediate visibility into sync status. This was also showcased in the GitLab 17.8 release post. Better release information: Implemented a clear view of deployments related to a release, improving deployment tracking and visibility. Streamlined environment search: Fixed an issue where users couldn't effectively search the Environments page, improving navigation in large environment lists. Enhanced error message display: Resolved issues with viewing Flux details when long error messages were present, making troubleshooting more straightforward. Looking forward The success of these improvements demonstrates the value of empowering our teams to make direct, meaningful changes to our experience. Beyond the product enhancements, one of the most valuable outcomes has been the strengthened relationship between our Frontend and Design teams. Working together closely on these improvements has fostered better understanding of each other's perspectives, workflows, and constraints, leading to more effective collaboration. This deepened partnership has created a foundation for even better collaboration in our regular workflow, as team members now have stronger working relationships and shared understanding of each other's domains. We're excited to continue this initiative in future iterations, not just for the product improvements it generates, but also for its role in building stronger, more cohesive teams. Follow along with the "Beautifying our UI" project as we continue to make improvements to GitLab. Read more How we overhauled GitLab navigation GitLab dark mode is getting a new look Beautifying our UI: Giving GitLab build features a fresh look
In early 2024, we started a journey to implement better metrics for our internal Red Team. Our first iteration focused on what we now call the adoption rate metric, which measures how often the recommendations our team makes are accepted and implemented. Choosing this metric was very deliberate. While there are many ways to measure a Red Team's impact, we wanted to start with something fundamental: Are we actually driving meaningful security improvements? The adoption rate directly ties our work to real security outcomes, and we could measure it using tools and processes we already had in place. In this article, you'll discover how we used GitLab to track these results end-to-end, some lessons we learned (including what we would have done differently), and our plans to tackle the next set of metrics. How we implemented the adoption rate metric We use GitLab extensively for our Red Team planning, execution, and reporting. Every operation wraps up with a report that's written in markdown in a dedicated GitLab project. Each report contains a section called "Recommendations" with a list of suggestions to make GitLab more secure. Those recommendations are always linked to a dedicated issue, which we open in the project closest to the team who can address it. If we're suggesting a product feature, it goes directly in that tracker. If it's a detection capability, it goes into the detections as code repository. We always assign a directly responsible individual (DRI) in the group that owns that space, and we use this issue template to ensure consistency in describing the problem, the risk, and potential solutions. Here's where the tracking logistics come in. We use GitLab labels to classify the recommendation across three categories: Detections and alerts (RTRec::Detection) Security controls (RTRec::Control) Processes and procedures (RTRec::Process) We then use another set of labels to follow the lifecycle of that recommendation – from review all the way through adoption: Under review (RecOutcome::UnderReview) Accepted and actively being worked on (RecOutcome::InProgress) Accepted but backlogged (RecOutcome::Backlogged) Accepted but blocked (RecOutcome::Blocked) Fully adopted and closed (RecOutcome::Adopted) Partially adopted and closed (RecOutcome::PartiallyAdopted) Not adopted and closed (RecOutcome::NotAdopted) How we stay on top of recommendations We use a new GitLab feature called "GitLab Query Language" (GLQL) to build a dynamic Security Recommendations Dashboard inside a GitLab issue. This issue allows us to quickly identify things like: open recommendations that haven't been updated recently open recommendations that have been backlogged for an extended period of time closed recommendations that weren't properly labeled with an adoption outcome We've found this process encourages the Red Team to follow up on stale recommendations, reaching out to the owners and seeing how we can help get them adopted. GLQL is very cool, and allows us to turn a short code block like this: --- display: table fields: title, labels("RTRec::*"), labels("RecOutcome::*"), created, updated --- group = "gitlab-com" AND label = "RTRec::*" AND opened = true ... into a dynamic table like this: That table for us is very tactical and we use it to keep things moving. Beyond that, we also visualize the adoption rate trends over time. That allows us to look at things like quarterly adoption rate percentages, how long different types of recommendations take to adopt and implement, and how these figures vary across departments. Lessons learned 1. Start with metrics in place; don't wait for your program to mature first. Early in our Red Team's development, we focused more on how we would execute operations and less on how we would measure them. The idea of using metrics to distill complex operations into simple numbers felt like it might oversimplify our work. But we've learned that thoughtful metrics don't reduce the value of Red Team operations - they help demonstrate our impact and guide our program's growth. Starting with clear metrics earlier would have accelerated this growth. Implementing these metrics later meant spending significant time reformatting years of historical recommendations to enable consistent analysis. Had we planned for metrics from the start, we could have saved ourselves a lot of time and effort. We’re keeping this lesson in mind as we start on our next set of metrics, threat resilience, which we talk about below. 2. Don't operate in a silo. Red Teams aren't the only groups that provide recommendations in a security organization. At GitLab, we have our bug bounty program, our external pentests, product security, security assurance, and security operations. On the Red Team, we developed our own recommendations process from scratch. It's been fairly effective, but we have noticed some areas for improvement, particularly around prioritization, project management, and alignment with our organization's risk reporting process. We also noticed that some other teams are really good at these areas such as our bug bounty program and the triaging of findings from our external pentests. Those particular groups are very good at delivering product recommendations, and we've been learning from their approach to improve our own delivery methods. So we've taken our success with visualizing metrics and are integrating these lessons to create a more standard format that can be used across teams. This will allow us to leverage things that are working well, like our adoption rate metric, and combine them with the more efficiently managed processes used by other groups to ultimately achieve a higher adoption rate and a more secure GitLab. Next up: Measuring our threat resilience Next up for us is implementing metrics around threat resilience. We want to measure how well GitLab can prevent, detect, and respond to the threats most relevant to our organization. We're building a dashboard that will help visualize this data, showing our top threat actors and a series of scores that measure how well we defend against their specific techniques. Our goal is to have this dashboard drive decisions around what Red Team operations to conduct, what defensive capabilities to improve, and in general where we should be investing time and effort across our entire security division. We hope to consolidate our existing tools in this process and are currently evaluating solutions. We'll share more info when we've achieved some success here. Key takeaways and how to get started If you're looking to measure your Red Team's impact, here's what we've learned: Start tracking metrics early, even if they're not perfect. Focus on actionable metrics first (like adoption rate). Use your existing tools. We used GitLab and Tableau, but the approach works with any tracking system. Collaborate across security teams to leverage existing processes when possible. We'd love to hear about your experience with metrics in security so drop a comment below or open an issue in one of our public projects. Read more from GitLab's Red Team Stealth operations: The evolution of GitLab's Red Team How GitLab's Red Team automates C2 testing How we run Red Team operations remotely
Today we’re excited to announce the launch of GitLab’s Open Source Security Hub — a central repository of security-focused projects developed by GitLab’s internal security team. These tools are designed to help developers, security practitioners, and organizations build safer, more secure software, and more resilient security programs. Securing systems is an ongoing challenge for businesses as threat actors continually adapt to new technologies and find creative ways to exploit organizations. Not only are they evolving their tactics, techniques and procedures, but they’re also collaborating through criminal networks, sharing strategies, stolen data, and malicious tools to launch coordinated attacks at scale. As these threats grow in complexity, community-driven collaboration is one of our most powerful defenses. It’s a notion we’ve long understood in security — that defending against adversaries is a shared responsibility. By working together as a community, we can accelerate our collective intelligence and stay ahead of adversaries. In open-sourcing our security solutions, we aim to empower teams to adapt faster, respond smarter, and defend better — together. Why open source security? At GitLab, open source isn’t just part of our technology — it’s part of our founding story. Since day one, we’ve championed the open source philosophy, believing that transparency, collaboration, and community-driven development are keys to building better software. Over the years, GitLab has fostered an open source community with more than 4,000 contributors and has provided a comprehensive DevSecOps platform through its open source Community Edition. We’ve also been inspired by industry leaders like Crowdstrike and Palo Alto Networks, who have shown that open-sourcing security tools not only improves innovation but also strengthens the entire security ecosystem. Following in their footsteps, GitLab is committed to supporting the community by sharing tools, templates, and frameworks developed by our security teams. Explore our featured open source security projects We’re launching the Open Source Security Hub with a range of projects designed to enhance security operations and risk management. Here are some of the featured projects: StORM templates: Streamline your security risk program with templates that standardize risk tracking and reporting. GUARD Framework: Automate response and detection with a detections-as-code approach that simplifies detection creation, maintenance, and alert routing. GitLab CIS Benchmark Scanner: Improve your project’s security posture by auditing against the Center for Internet Security GitLab Benchmark. Whether you’re a security engineer, researcher, or developer, your expertise and contributions are invaluable. Join us in strengthening the security ecosystem and collaborating with a community dedicated to making software safer for everyone. Explore GitLab’s Open Source Security Hub and contribute to the next chapter of open source security. Learn more New CIS GitLab Benchmark scanner boosts security and compliance GitLab introduces new CIS Benchmark for improved security Unveiling the GUARD framework to automate security detections at GitLab Automating cybersecurity threat detections with GitLab CI/CD
As companies rapidly adopt AI technologies, CISOs face a new frontier of security challenges. Many security leaders find themselves grappling with unfamiliar questions: How do we evaluate AI vendors differently from traditional software vendors? What security controls matter most? Where does vendor responsibility end and customer responsibility begin? How do we evaluate AI security risks within the context of the service provided? To help answer these questions, we’ve created the GitLab AI Security Framework to show security leaders how GitLab and customers enable secure AI-powered development using GitLab Duo. The genesis of AI security challenges From conversations with security leaders across industries a pattern has emerged: Organizations are rapidly embracing AI technologies to improve delivery while their security teams struggle to establish appropriate security controls. This disconnect isn't just a matter of resources or expertise – it represents a fundamental shift in how organizations need to approach security in the AI era. Security leaders are witnessing quick and unprecedented adoption of AI across their organizations, from development teams using coding assistants to marketing departments leveraging generative AI. While organizations are integrating AI within their own software, many of their current vendor-provided SaaS applications have added AI capabilities as well. Although this adoption drives innovation and efficiency, it also creates a complex set of security considerations that traditional frameworks weren't designed to address. Below are some of the specific challenges we’ve identified. Security challenges in the AI era 1. Responsibility and control uncertainty The rapid pace of AI adoption has left many organizations without a coherent security governance strategy. Security teams find themselves trying to retrofit existing security frameworks to address AI-specific concerns. Security leaders face challenges in understanding where their responsibilities begin and end when it comes to AI security. The traditional vendor-customer relationship becomes more complex with AI systems, as data flows, model training, and inference processes create new types of interactions and dependencies. 2. Risk assessment evolution Traditional security risk models struggle to capture the unique characteristics of AI systems. Security leaders are finding that standard risk assessment frameworks don't adequately address AI-specific risks. AI security risks will differ based on AI implementation and the context in which it’s used. The challenge is compounded by the need to evaluate AI vendors without necessarily having deep technical AI expertise on the security team. 3. Data protection complexities 4. Compliance and standards navigation Addressing these challenges With the release of GitLab Duo, we recognized these executive-level concerns and developed a comprehensive framework to help organizations navigate AI security in the context of our AI-powered DevSecOps platform. Our AI Security Framework provides details on our privacy-first implementation of AI to enable GitLab Duo, and how we validate the security of our AI vendors. A responsibility matrix is included to help security leaders manage their AI security responsibilities while enabling their organizations to innovate safely. We also compiled a selection of AI-specific security risks to keep in mind and highlighted how GitLab capabilities like prompt guardrails can help in mitigating them. Want a deeper look at our security controls? Check out our AI Security Framework. Learn more GitLab AI Transparency Center How GitLab uses prompt guardrails to help protect customers Improve AI security in GitLab with composite identities Secure, compliant, and AI-powered: Get to know 3 new GitLab features ICYMI: Key AI and security insights from our developer community
Development environments often require the ability to build and run containers as part of their local development. Securely running containers within containers can be challenging. This article will provide a step-by-step guide to securely build and run containers in a workspace. You will learn how to: Create a Kubernetes cluster on AWS EKS Configure Sysbox Configure GitLab agent for Kubernetes and GitLab Workspaces Proxy Configure sudo access for a workspace with Sysbox Configure Ingress Controller Build containers inside a workspace Run containers inside a workspace Get started today Create a Kubernetes cluster on AWS EKS Install the AWS CLI on your local machine. Next, configure a named profile and export it to ensure all the following aws commands use the set credentials. aws configure --profile gitlab-workspaces-container-demo export AWS_PROFILE=gitlab-workspaces-container-demo Install eksctl, a CLI to interact with AWS EKS. Let’s now create a Kubernetes 1.31 cluster on AWS EKS with 1 node of Ubuntu 22.04 of c5.2xlarge instance type. The nodes can autoscale from 0-20 nodes and each node will have a label sysbox-install: yes . This will be explained later in the article. export CLUSTER_NAME="gitlab-workspaces-container-demo-eks-sysbox" eksctl create cluster \ --name "${CLUSTER_NAME}" \ --version 1.31 \ --node-ami-family=Ubuntu2204 \ --nodes=1 \ --nodes-min=0 \ --nodes-max=20 \ --instance-types=c5.2xlarge \ --node-labels "sysbox-install=yes" \ --asg-access \ --external-dns-access \ --full-ecr-access Create an IAM OIDC provider for your cluster. eksctl utils associate-iam-oidc-provider --cluster "${CLUSTER_NAME}" --approve Create IAM role for EBS add-on for EKS. eksctl create iamserviceaccount \ --name ebs-csi-controller-sa \ --namespace kube-system \ --cluster "${CLUSTER_NAME}" \ --role-name "AmazonEKS_EBS_CSI_DriverRole_${CLUSTER_NAME}" \ --role-only \ --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \ --approve Create Amazon EBS CSI driver add-on for Amazon EKS cluster. eksctl utils describe-addon-versions --kubernetes-version 1.31 | grep aws-ebs-csi-driver export AWS_ACCOUNT_ID="UPDATE_ME" eksctl create addon \ --cluster "${CLUSTER_NAME}" \ --name aws-ebs-csi-driver \ --version latest \ --service-account-role-arn "arn:aws:iam::${AWS_ACCOUNT_ID}:role/AmazonEKS_EBS_CSI_DriverRole_${CLUSTER_NAME}" \ --force Install kubectl, a command line tool for communicating with a Kubernetes cluster's control plane, using the Kubernetes API. Let’s get the kubeconfig of the created cluster. aws eks update-kubeconfig --name "${CLUSTER_NAME}" Configure Sysbox Sysbox is a container runtime that improves container isolation and enables containers to run the same workloads as virtual machines. Install Sysbox on the Kubernetes cluster using the sysbox-deploy-k8s daemonset. curl https://raw.githubusercontent.com/nestybox/sysbox/refs/tags/v0.6.6/sysbox-k8s-manifests/sysbox-install.yaml -o sysbox-install.yaml Because of how Sysbox releases itself, it first created a git tag, which runs a pipeline to build assets after which the YAML files for the sysbox-deploy-k8s daemonset are updated. Thus, we need to update the DaemonSet's spec.template.soec.containers[0].image to registry.nestybox.com/nestybox/sysbox-deploy-k8s:v0.6.6-0 . new_image_value="registry.nestybox.com/nestybox/sysbox-deploy-k8s:v0.6.6-0" temp_file=$(mktemp) sed -E "s|^([[:space:]]*image:)[[:space:]]*.*|\1 $new_image_value|" "sysbox-install.yaml" > "$temp_file" mv "$temp_file" "sysbox-install.yaml" Apply the YAML file to Kubernetes and ensure all the pods of the DaemonSet are running. kubectl apply -f sysbox-install.yaml kubectl get pod -A kubectl -n kube-system get daemonset Verify the installation by creating a pod which uses Sysbox container runtime. cat /dev/null sudo apt-get update sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin # Start the Docker Daemon sudo dockerd Build the container image. sudo docker build -t workspaces-golang-server . Run containers inside a workspace Let’s run the container built above and expose port 3000 from the container onto the host (workspace). sudo docker run -p 3000:3000 workspaces-golang-server The port 3000 is exposed in the .devfile.yaml used to create the workspace. Access the server running inside the container from the browser. Here is a video clip. Get started today From GitLab 17.4, you can build and run containers securely in GitLab Workspaces. See our documentation for more information. Replace your local development environments to GitLab Workspaces for a secure, ephemeral, reproducible development environment. Read more Enable secure sudo access for GitLab Remote Development workspaces Quickstart guide for GitLab Remote Development workspaces Contributor how-to: Remote Development workspaces and GitLab Developer Kit
A personal website is more than just a utility for digital creators and professionals in tech. It's a representation of your brand. But creating one from scratch can be time-consuming and expensive. With GitLab Pages, you can host your website with built-in features, including SSL certificates and a GitLab-provided domain. All of this is available on GitLab's free tier, making it an efficient solution for hosting your professional presence. We're going to take you on a fun journey to craft a stunning personal website using GitLab Pages! We’ve got a super simple, versatile template that you can easily jazz up to reflect your unique style. So grab your favorite snack, get comfy, and let’s turn your online presence into something truly fabulous! Prerequisites You will need the following prerequisites before getting started: A GitLab account (the free tier is sufficient) Basic familiarity with HTML/CSS Content and images you want to add to your website (optional) Once you’re set up with a GitLab account and have your content handy, you can move on to the next steps. Step 1: Create a new project Sign on to your GitLab account and create a project. Click Create blank project. Fill in your project details: Name your project yourusername.gitlab.io. Replace yourusername with your GitLab username. Tip: The project name determines your website’s URL. If you name your project yourusername.gitlab.io, your website will be available at https://yourusername.gitlab.io with no additional path. However, if you use any other project name, your site will be available at https://yourusername.gitlab.io/project-name. Make the project public. Click Create project. Step 2: Add the template files Start by creating two new files in your repository: First, create index.html: In your project, click the + button and select New file. Name the file index.html. Add your HTML content. Use the example HTML provided below. (Pro tip: Users can ask GitLab Duo Chat to generate HTML for enhanced functionality.) [Your Name] · [Your Title] Hello, I'm [Your Name] ( @[Your Handle] ) I'm a [Your Title] I'm a [Your Role] at [Your Company], [Brief company description]. Your personal statement about what you do and what you're interested in. Add your contact preferences here. — [Your Name] ( @[Your Handle] ) Add a commit message (e.g., "Added index.html"). Click Commit changes. Create style.css (follow same steps above). body { margin: 0; padding: 0; background: #000; color: #f4f4f4; font-family: "Graphik Web", system-ui, -apple-system, BlinkMacSystemFont, "Helvetica Neue", "Helvetica", "Segoe UI", Roboto, Ubuntu, sans-serif; font-weight: 400; font-smooth: antialiased; -webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; } a { color: #ff310a; text-decoration: none; } a:hover { color: #CFEF54 } .content { max-width: 40rem; margin: 0 auto; } img.photo { border-radius: 50%; } p { font-size: 1.5rem; line-height: 1.4; margin: 0; letter-spacing: -0.05rem; } h2 { font-weight: 400; line-height: 1.3; letter-spacing: -0.05rem; } .smallcaps { font-variant: small-caps; color:#333; } .gray{ color: #999; } .preloader { display: flex; justify-content: center; align-items: center; height: 100vh; height: -moz-available; height: -webkit-fill-available; height: fill-available; width: 100%; background: #000; position: fixed; top: 0; left: 0; z-index: 9999; transition: opacity 0.3s linear; transform: translate3d(0, 0, 0); } body.loaded .preloader { opacity: 0; } .fade { animation: fadeIn 1s ease-in-out both; } .fade:nth-child(2) { animation-delay: 1s; } .fade:nth-child(3) { animation-delay: 2s; } .fade:nth-child(4) { animation-delay: 3s; } .fade:nth-child(5) { animation-delay: 4s; } .fade:nth-child(6) { animation-delay: 5s; } .fade:nth-child(7) { animation-delay: 6s; } .fade:nth-child(8) { animation-delay: 7s; } .fade:nth-child(9) { animation-delay: 8s; } .fade:nth-child(10) { animation-delay: 9s; } .fade:nth-child(11) { animation-delay: 10s; } .fade:nth-child(12) { animation-delay: 11s; } .fade:nth-child(13) { animation-delay: 12s; } @keyframes fadeIn { from { opacity: 0; transform: translate3d(0, 0%, 0); } to { opacity: 1; transform: translate3d(0, 0, 0); } } Step 3: Configure GitLab CI file There are two ways to create the GitLab CI configuration file that tells GitLab how to build and deploy your site: Option 1: Use Pipeline Editor (recommended) Go to your project's Build > Pipeline Editor. The .gitlab-ci.yml file will be automatically created. Copy and paste the following configuration: pages: stage: deploy script: - mkdir .public - cp -r * .public - mv .public public artifacts: paths: - public only: - main Option 2: Manual creation If you prefer to create the file manually: Create a new file named .gitlab-ci.yml. Add the following configuration: pages: stage: deploy script: - mkdir .public - cp -r * .public - mv .public public artifacts: paths: - public only: - main The key to getting your site running is the GitLab CI configuration file. This file tells GitLab how to build and deploy your site. Let's break down what each part does: The script part script: - mkdir .public - cp -r * .public - mv .public public This creates a folder called public and copies all your website files into it. GitLab Pages uses this folder to serve your website by default, though you can customize the publishing folder if needed. The only part only: - main This tells GitLab to only update your website when changes are made to the main branch. This helps prevent accidental updates from experimental changes. Step 4: Watch the magic happen Commit all your changes. Go to Build > Pipelines to watch your deployment. Wait for the pipeline to complete successfully (indicated by a green checkmark). Step 5: Access your website Once the pipeline completes successfully, your website will be available at: https://[yourusername].gitlab.io/ . You can find an overview of your deployed website and additional settings in your project's Deploy > Pages section. Here you'll find useful information. including: Your website's access URLs Domain settings By default GitLab enables Unique domain. Make sure to disable it if you want to use the GitLab-provided domain. Learn more with the [unique domain documentation](https://docs.gitlab.com/ee/user/project/pages#unique-domains]. HTTPS certificates status Recent deployments Additional configuration options Custom domains This section is particularly helpful when setting up custom domains or troubleshooting deployment issues. Customize your site Replace all “Your ...” placeholders in index.html with your information. Add your images: profile.png - your profile photo (64x64px) favicon.png - your site favicon (32x32px) Og.png - OpenGraph image for social media preview (1200x630px) See it in action If you're familiar with GitLab, feel free to fork my repository to get started quickly. Here is the final result: https://fracazo.gitlab.io/ Common issues and solutions By default, GitLab enables "Unique domain" for Pages projects. To use the simpler GitLab-provided domain (like username.gitlab.io), go to Deploy > Pages and disable the "Use unique domain" option. While unique domains offer some technical advantages, like better asset path handling, you might prefer the cleaner URL structure for a personal website. If your pipeline fails, check that you're using main instead of master in your .gitlab-ci.yml file. Ensure your group and project is public for GitLab Pages to work. If any jobs fail in your pipeline, you can check the job log for detailed error messages to help with troubleshooting. With GitLab Pages and this template, you can have a professional/personal website up and running in minutes. The template is clean, responsive, and easy to customize. As you grow professionally, you can easily update your site directly through GitLab. You can automate the deployment process by leveraging GitLab's CI/CD capabilities and focusing on creating great content. The best part? All of this is available on GitLab's free tier, making it an excellent option for free hosting of your personal projects, documentation sites, or even small business websites. For more advanced features and configurations, check out our Pages documentation. What’s next for GitLab Pages? We're constantly working to make GitLab Pages even better for creators and developers. Here are some exciting improvements coming soon: Simplified domain management We have some exciting updates coming to GitLab Pages that will make managing your domains even easier and more fun! You can look forward to a streamlined dashboard that brings all your domain settings together in one friendly space, making everything easily accessible. You’ll stay informed with real-time updates on your DNS and SSL certificate statuses, helping you keep your domains secure and running smoothly. Custom domain setup Setting up custom domains will be a breeze with our easy-to-follow process, guiding you every step of the way. Plus, you'll be able to set up your custom domains to automatically redirect visitors from your old website address to your new one – perfect for when you want all your traffic to go to one main website. Learn more about custom domains. Get started with GitLab Pages today with GitLab's free tier! Learn more GitLab Pages features review apps and multiple website deployment Pages: Multiple website deployment documentation GitLab Pages examples
We are excited to announce the general availability of GitLab Duo Self-Hosted for Code Suggestions and Chat. An optional capability for self-managed customers with a GitLab Duo Enterprise subscription, GitLab Duo Self-Hosted supports deployment flexibility across multiple platforms, including on-premises infrastructure or in private clouds and secure cloud environments through AWS Bedrock and Azure OpenAI. GitLab Duo Self-Hosted empowers teams to innovate with AI while helping them maintain control over sensitive data and intellectual property. Security concerns have been a major barrier to AI adoption in regulated industries. In our Global DevSecOps Survey, more than half of the respondents said that introducing AI into the software development lifecycle is risky. With GitLab Duo, we gave organizations a way to ship more secure software faster with AI throughout the entire software development lifecycle. GitLab Duo Self-Hosted expands the availability of GitLab Duo AI features to organizations with stringent data privacy requirements, offering flexibility in both AI large language model (LLM) selection and deployment options. The earliest adopters of GitLab Duo Self-Hosted include organizations in the public sector and regulated industries – e.g., financial services, automotive, and healthcare. These organizations seek to gain the competitive advantage of AI by integrating AI-powered development tools into their environments, while also giving security teams the control they need. As one U.S. government agency says: “After selecting GitLab as the cornerstone of our agency-wide DevSecOps platform, we chose GitLab Duo Self-Hosted to further advance our software factory capabilities. GitLab Duo’s ability to operate in air-gapped environments and provide granular control over our data was crucial to delivering secure AI-powered features. This unified approach streamlines our workflow and strengthens security, allowing us to leverage AI for increased productivity while meeting strict compliance requirements.” Architect secure AI deployments GitLab Duo Self-Hosted enables GitLab Duo features that leverage a curated selection of leading AI LLMs, including those from Anthropic, Mistral, and OpenAI. Here are the LLMs supported by GitLab today: On-premises - Mistral models with the vLLM serving platform AWS - Mistral and Anthropic Claude 3.5 Sonnet via AWS Bedrock Microsoft Azure - OpenAI GPT models via Azure AI We are evaluating more models to support in the near future. Learn more about the LLMs we support. GitLab Duo Self-Hosted deployment options include on-premises installations powered by the open-source vLLM framework, as well as private-cloud deployments via services like AWS Bedrock and Microsoft Azure AI. This flexibility helps organizations to architect AI solutions that align with their unique security, compliance, and performance requirements. Simplify AI/ML implementation GitLab Duo's AI abstraction layer standardizes and simplifies the integration of the chosen LLM to a feature, mitigating the burden of implementing AI/ML technologies. This enables companies to streamline their AI adoption efforts and enhance the developer experience, free from the complexities of integrating and maintaining multiple tools. Maintain control of sensitive data By isolating your GitLab instance, AI gateway, and LLMs in your own environment or country of choice, GitLab Duo Self-Hosted makes it possible that sensitive data and intellectual property remain within your designated perimeter. Granular control over data locality helps enable adherence to strict data residency regulations, while adopting AI capabilities in secure settings. Whether you use GitLab Duo Self-Hosted in a completely air-gapped environment with vLLM or leverage a supported private cloud, you can control all aspects of the deployment to include the geographic location of components. By eliminating the reliance on external APIs and providing full visibility into all request and response logs, GitLab Duo Self-Hosted helps even the most regulated organizations confidently adopt AI capabilities and meet the most stringent compliance obligations. Start an interactive tour of GitLab Self-Hosted by clicking on the image below: Get started with GitLab Duo Self-Hosted today If you're ready to advance your AI journey while addressing security and data privacy, reach out to us to help set up GitLab Duo Self-Hosted in your environment today.
Software development environments can be complex to set up and maintain. Developers often spend a significant amount of time configuring their local environments with the right dependencies, tools, and settings. GitLab aims to solve this by providing a default devfile that enables you to create workspaces and to start developing quickly. GitLab Workspaces GitLab Workspaces provide isolated development environments for making changes to your GitLab projects without the complexity of setting up local dependencies. Workspaces ensure reproducible development setups, allowing developers to share their environment configurations effortlessly. By default, GitLab Workspaces are configured to use the GitLab VS Code fork and include the GitLab Workflow extension. To learn more, visit the GitLab Workspaces documentation. Understand devfiles A devfile is a YAML-based declarative configuration file that defines a project's development environment. It specifies the necessary tools, languages, runtimes, and other components required for development. Previously, setting up a workspace required a custom devfile at the root of the repository. For example, a .devfile.yaml file. A typical devfile looked like this: GitLab default devfile Starting in GitLab 17.9, a GitLab default devfile is available for all projects when creating a workspace. This eliminates the need to manually create a devfile before starting a workspace. Here is the content of the default devfile: When creating a workspace with the GitLab UI, the option Use GitLab default devfile is always available – regardless of whether custom devfiles exist in the repository. Simply select this option to start exploring GitLab Workspaces with one less setup step. Create your own custom devfiles While the GitLab default devfile provides a quick way to start a workspace, you may want to customize your development environment to better fit your project's needs. By creating a custom devfile, you can tailor your development environment with the exact tools, dependencies, and configurations needed for your workflow. Consider creating a custom devfile if you need to: Add project-specific dependencies beyond the base development image. Adjust CPU and memory resource limits. Configure multiple containers for additional services like databases. Define custom, project-specific, environment variables. Set up specific port mappings. Integrate specialized development tools like debuggers or language servers. For more details, see the Workspaces devfile documentation. Read more Use GitLab AI features out-of-the-box in a GitLab Workspace Quickstart guide for GitLab Remote Development workspaces Enable secure sudo access for GitLab Remote Development workspaces
Imagine this: You are an engineer at a growing tech company, and it’s 2 a.m. when you get an urgent call. A critical deployment pipeline has failed, and your team is scrambling to figure out why. After hours of digging, you realize someone revoked a personal access token belonging to an engineer who left the company a week ago. This token was tied to several key automation processes, and now your entire system is in chaos. How do you make sure it does not happen again? Follow this guide, which takes GitLab customers through the end-to-end process of identifying, managing, and securing their tokens. It is meant to be a handy supplement to the extensive token overview documentation for GitLab administrators, developers, and security teams who need to ensure proper token management within their projects. Here's what is covered in this guide: How to select the right token for the job Token types Discovering your tokens Credentials inventory Managing tokens in the GitLab UI and API Token rotation and expiration management Token management best practices Service accounts How to select the right token for the job Choosing the right token ensures optimal security and functionality based on your use case. Tokens can be used for authenticating API requests, automating CI/CD pipelines, integrating third-party tools, managing deployments and repositories, and more. For the sake of simplicity, the chart illustrates a straightforward use case tied to single user ownership. For more information, check out our documentation of user roles and permissions at each namespace (user/group) in your instance or top-level group. Example use cases could be as follows: Personal access tokens (PAT) can be used by developers when a user's personal access and permissions are required. In this case, the credentials follow the status and permissions of the user, including the removal of access if the account loses access to a specific project or group (or is blocked entirely). Project/group access tokens (PrAT/GrAT) are recommended when access should be scoped to resources within a specific project/group, allowing anyone with a PrAT/GrAT to access those resources through mechanisms managed by assigned scopes. Token types Below is a list of GitLab tokens with their default prefixes and use cases. For more information, please visit the GitLab Token overview page. Tokens Prefix Description Personal access token glpat Access user-specific data OAuth 2.0 Token gloas Integrate with third-party applications using OAuth2.0 authentication protocol Impersonation token glpat Act on behalf of another user for administrative purposes Project access token glpat Access data from a specific project Group access token glpat Access data from a specific group Deploy token gldt Clone, push, and pull container registry images of a project without a user and a password Deploy keys N/A Allow read-only or read-write access to your repositories Runner authentication token glrt Authenticate GitLab Runners CI/CD job token glcbt Automate CI/CD processes Trigger token glptt Trigger pipelines manually or programmatically Feed token glft Authenticate access to package/RSS feeds Incoming mail token glimt Process incoming emails GitLab agent for Kubernetes token glagent Manage Kubernetes clusters via the GitLab agent SCIM tokens glsoat Enable SCIM integrations for user provisioning Feature flags client token glffct Enable feature flags programmatically Webhook token N/A User set secret token to secure webhook payloads and ensure that the requests are from GitLab Discovering your tokens Credentials inventory On GitLab Ultimate, administrators (GitLab Self-Managed) and top-level group owners of an enterprise organization (GitLab.com as of Version 17.5) can monitor the credentials in their namespace. This inventory tracks token details such as: Token type Available tokens on GitLab.com Available tokens on GitLab Self-Managed Associated user accounts Token scopes, and creation and expiration dates Token last used IP addresses (as of GitLab 17.10) Token filtration based on the above user-defined parameters Ability to revoke and rotate those tokens A well-maintained credentials inventory helps identify over-permissioned tokens, and gives insight into credentials that may need to be rotated, ensuring a secure and efficient workflow. Credentials inventory API As a complement to the UI, there is ongoing development to release a credentials inventory API through the new /group/:id/manage endpoint. The credentials accessible under this endpoint are limited to enterprise users, and can be accessed by the top-level group owner of an enterprise organization. An example of the future API call would be: curl --header "PRIVATE-TOKEN: " "https://verified_domain.com/api/v4/groups//manage/personal_access_tokens" GitLab API The GitLab API allows you to programmatically list and manage tokens within your organization. Key authentication-related endpoints support various token types), including personal, group, CI/CD tokens, and more. An example of using a personal access token to list all visible projects across GitLab for the authenticated user is: curl --header "PRIVATE-TOKEN: " \ "https://gitlab.example.com/api/v4/projects" Watch this video to learn how to make API calls to the GitLab API. Finding where tokens are used Customers can find where tokens are used in different ways: under User Profile > Access Tokens in credentials inventory in audit events via the API Information on token usage is updated every 10 minutes for last_used and every minute for last_used_ip. The ability to view IP addresses was introduced in GitLab 17.9, and is controlled by the :pat_ip feature flag. Follow these steps to view the last time a token was used, along with its last five distinct IP addresses. Managing tokens in the GitLab UI and API The following table includes videos detailing a few token creations in the UI and demonstrates their usage via the API. Tokens GitLab UI GitLab API Personal access token Documentation and video Documentation and video Group access token Documentation and video Documentation and video Project access token Documentation and video Documentation and video Token rotation and expiration management Implementing token rotation and strict expiration policies reduces the risk of compromise and ensures compliance with security standards. Regular rotation and enforced expirations prevent stale credentials from becoming security vulnerabilities. Previously, expired group and project access tokens were automatically deleted upon expiration, which made auditing and security reviews more challenging due to the lack of a record of inactive tokens. To address this, a recent feature introduced the retention of inactive group and project access token records in the UI for 30 days after they became inactive. This enhancement aims to allow teams to track token usage, expiration, and revocation for better compliance and monitoring. To be more proactive in your token rotation and expiration management, do the following: Actively rotate your tokens via the UI or API. If you use the latter, be mindful of the automatic token reuse detection security mechanism. Set an instance-wide maximum lifetime limit for access tokens. Token rotation API Until GitLab 17.7, customers had to programmatically rotate access tokens with the API. Its counterpart is now available on the UI. Check out the video in the table below or follow the documentation for guidance. Token rotation snippets The following table includes videos detailing the rotation of GitLab tokens. Tokens Prerequisites GitLab UI GitLab API Personal access token Scope: api Documentation and video Documentation and video Group access token Scope: api and Role(s): owner Documentation and video Documentation and video Project access token Scope: api and Role(s): owner, maintainer Documentation and video Documentation and video Token management best practices Principle of least privilege Mitigate risk by restricting assigned permissions to tokens required for their respective tasks. This allows you to proactively predict and troubleshoot points of failure in your systems. You can do this by: Selecting the right token for the right job. See the flowchart. Assign only the required scopes when creating a token. For example, use read-only scopes for tokens with auditor-like jobs. See roles. Avoid granting administrative privileges unless specifically required. Enforce instance-wide default token lifetimes. Regularly review and audit token permissions to ensure they align with current operational needs. Revoke tokens once the task is complete. Service accounts Service accounts ensure tokens are tied to non-human entities, separating them from individual user accounts and reducing dependency on specific users. Instead of using personal accounts to generate tokens for automation, create service accounts with limited scopes. Benefits include: Usage of service account tokens in CI/CD pipelines to avoid disruptions caused by user account changes Programmatically automate rotation processes, as personal accounts remain unaffected Clearer monitoring and auditing trail of actions taken by service accounts Service accounts with no expiration date Does not consume a license seat GitLab plans to release a new Service Accounts UI as a counterpart to its API-based creation, designed to simplify the management of service accounts and their associated tokens. Check out the demo below on the programmatic usage of service accounts. Vulnerability tools Leverage GitLab’s built-in security tools to identify and mitigate vulnerabilities associated with token usage. For maximum coverage, it is recommended to use them all in tandem. Secret Detection: Scans your repository for hardcoded secrets like API tokens, passwords, and other sensitive information. View the list of detected secrets. Static Application Security Testing (SAST): Analyzes your source code for security vulnerabilities and provides reports with UI findings in merge requests, among other features. Dependency Scanning: Ensures that third-party libraries used in your project do not expose token-related vulnerabilities. Audit logs and monitoring Maintain token health by regularly reviewing audit logs and token usage, instance- and/or group-wide. Audit events: Enable audit event logging in GitLab to track token-related activities such as creation, usage, deletion and unusual API calls (unpermitted parameters in logs, and consistent triggers of the rate limiter). IP allowlisting: Helps prevent malicious users from hiding their activities behind multiple IP addresses. Alerts: Set up alerts for unusual activities (trigger paging for on-call rotations or be used to create incidents). Credentials inventory: Complete control of all available access tokens with the ability to revoke as needed. Notifications: Proactively handle any token (group, project, and personal) expiration notification emails you receive. Based on customer demand, this feature was recently extended to include 30-day and 60-day notifications from the seven-day default. Webhooks: Access token webhooks can be configured on groups and projects to send seven-day token expiry events. This feature was also recently extended to include 30-day and 60-day notifications behind the :extended_expiry_webhook_execution_setting feature flag (disabled by default). What's next With GitLab’s large token catalog, there are ongoing plans for consolidation with a focus on the lifetime, fine-grained scopes, consistent management, and usage. Our current prioritized token-related features include a complete UI for service accounts, additional credential types in the credentials inventory, and improved auditing for tokens and service accounts. Sign up for a free 60-day trial of GitLab Ultimate to start using token management.
Today, we're excited to announce the opening of the waitlist for the private beta of GitLab Duo Workflow: agentic AI built on top of the most comprehensive DevSecOps platform. The next step in our AI roadmap, GitLab Duo Workflow will help development teams navigate everything from security reviews to deployment processes, from debugging issues to cross-team coordination, all within the IDE. GitLab Duo Workflow leverages the structure for collaboration, continuous integration, continuous deployment, security, and compliance from the GitLab platform, to help organizations as they accelerate their development process with AI agents. Use GitLab Duo Workflow to help you: bootstrap a new development project modernize code perform contextual tasks create documentation enhance test coverage and more This is just the beginning. With GitLab’s unified data store, the more you use GitLab, the more context GitLab Duo Workflow has about your code, configurations, security findings, and deployment practices. The result: an increasingly powerful development experience that's tailored to your organization. The promise and challenge of AI agents Software has fundamentally changed the world, but only a tiny fraction of the world's population has the skills to build software today. Yet, these developers reach billions of people with smartphones and internet connections. Just imagine a world where more people can build, secure, and deliver production-ready software – there will be an explosion of innovation as more people can create software that impacts billions. Agentic AI will make that happen. AI agents understand context, maintain knowledge of entire codebases, and actively collaborate on complex software projects across development, security, and operations. With AI agents, developers can create software at a scale previously unimaginable for individuals or even teams. But this shift raises important questions about visibility, control, and how AI will impact developers' work. Organizations need to ensure AI enhances their developers' capabilities while enabling them to maintain oversight of their development process. The key to success isn't just adopting AI – it's adopting it in a way that empowers developers while preserving security, compliance, and governance. AI's success depends on your platform, not more add-on tools When you're working with more developers, code, and potential security risks, adding separate tools for each new challenge only creates more complexity. Our most recent DevSecOps Survey shows just how serious this problem is: DevSecOps teams are juggling up to 14 different tools, with professionals spending up to 80% of their time on non-coding tasks. For AI to be truly effective, it also needs high-quality, unified data. That's hard to achieve with disparate tools. The GitLab DevSecOps platform combined with GitLab AI agents brings everything together in a single data model that encapsulates source code, merge requests, epics, users, access rights, security policies, and more. The agents we're building use context about users and projects to standardize how teams work and automate the non-coding tasks that absorb developer time, such as scanning for security issues and enforcing compliance rules. When AI is built directly into the platform, these capabilities become even more powerful, turning AI agents into development partners while keeping you in control of how AI enhances the process. This isn't a far-off future — it's what we're building right now with GitLab Duo Workflow. ` GitLab Duo Workflow: AI agents on the most comprehensive DevSecOps platform Leveraging GitLab's end-to-end DevSecOps platform, GitLab Duo Workflow helps developers work at their highest potential. While AI coding assistants help with individual pieces of code, GitLab Duo Workflow will understand your entire development lifecycle – automating routine tasks so developers can focus on strategic innovation and creative problem-solving. As we develop GitLab Duo Workflow, here’s what it will be able to help teams achieve: From slow project setup to a running start Developers spend precious time configuring new projects, managing dependencies, and setting up basic infrastructure instead of building new features. With GitLab Duo Workflow, you can automate project bootstrapping directly in the IDE, providing the right configurations from the start so you can focus on innovation sooner. From legacy code to modern applications Modernizing legacy code is more than just updating syntax — it requires understanding dependencies, tests, CI/CD pipelines, and documentation. GitLab Duo Workflow helps modernize your codebase by handling code refactoring and streamlining all dependencies – from code to tests to CI/CD pipelines. From context switching to flow state Today, developers constantly switch between tools, docs, and codebases to solve problems. GitLab Duo Workflow will help resolve tasks with the full context of your codebase-related issues and merge requests, letting developers stay in their flow. From stale docs to dynamic knowledge Documentation becomes stale quickly, making codebases harder to understand and maintain. GitLab Duo Workflow supports developers in generating and updating documentation, including README files, code flow diagrams, and architecture documentation. From patchy to comprehensive testing As codebases grow, maintaining comprehensive test coverage becomes increasingly challenging. GitLab Duo Workflow can generate tests for entire sections of your codebase while integrating with your existing test infrastructure, ensuring more reliable software with less effort. Sign up for the private beta waitlist Sign up for the GitLab Duo Workflow private beta waitlist to see the next step in our vision for secure agentic AI – from project setup to security reviews to deployment. Built on GitLab's DevSecOps platform, these agents understand your entire software lifecycle while maintaining the enterprise-grade security and control organizations require.
At GitLab, we're passionate about using our own products internally, a.k.a. dogfooding. Dogfooding has led to significant improvements in accelerating our software delivery cycle time for customers. This article spotlights a specific use case where GitLab Value Stream Management (VSM) has driven significant improvements for our engineering team. You'll learn how VSM helped us tackle two critical challenges: measuring the journey from idea conception to merge request completion, and streamlining our deployment workflows. The Challenge: Identifying bottlenecks in MR reviews Despite having well-defined workflows, one team noticed that MRs were taking longer than expected to be reviewed and merged. The challenge wasn’t just about the delays themselves, but about understanding where in the review process these delays were happening and why. Our team’s goal was clear: Identify where time was being spent from the initial idea to the final merge of an MR. Pinpoint specific bottlenecks in the review process. Understand how MR size, complexity, or documentation quality affect review time. The Approach: Measures the MR review time in GitLab Value Stream Analytics Value Stream Analytics (VSA) enables organizations to map their entire workflow from idea to delivery, distinguishing between value-adding activities (VA) and non-value-adding activities (NVA) in the process flow. By calculating the ratio of value-added time to total lead time, the team can identify wasteful activities resulting in delays in MR reviews. To obtain the necessary metrics, the team customized GitLab VSA to gain better visibility into our MR review process. 1. Setting up a custom stage for MR review The team added a new custom stage in VSA called Review Time to Merge to specifically track the time from when a reviewer was first assigned to when the MR was merged. Start event: MR first reviewer assigned End event: MR merged By defining this stage, VSA began measuring the duration of the MR review process, giving us precise data on where time was being spent. 2. Using the Total Time Chart for clarity With the custom stage in place, the team used the Total Time Chart on the VSA Overview page (Analyze > Value Stream) to visualize how much time was spent during the new MR Review stage. By comparing the values represented by each area on the chart, the team could quickly identify how this stage contributed to the total software delivery lifecycle (SDLC) time. 3. Drilling down for deeper insights To investigate specific delays, the team used the Stage Navigation Bar to dive deeper into the MR Review stage. This view allowed them to: Sort MRs by review time: The stage table showed all related MRs, sorted by review duration, making it easy to detect slow MRs. Analyze individual MRs: For each MR, that team could examine factors such as reviewer assignment delays, multiple rounds of feedback, idle time after approval, and MR size/complexity. The outcome: Actionable insights and improvements By customizing VSA to track MR review time, the team uncovered several key insights: Delays in reviewer assignment: Some MRs experienced delays because reviewers were assigned late, or reviewers had too many MRs in their queue. Slow review start times: Even after assignment, certain MRs sat idle before reviews began, often due to context switching or competing priorities. Multiple feedback loops: Larger MRs often required multiple rounds of feedback, which extended review time significantly. Idle time post-approval: Some MRs were approved but not merged promptly, often due to deployment coordination issues. For the engineering manager on the team, VSA proved to be transformational/valuable in managing their team's workflow: "I've used the VSA to justify where we were spending time in MR completion. We have VSA customized to our needs, and it's been very beneficial to our investigations for opportunities for improvements.” Also, from this dogfooding experience, we’re now developing a key enhancement to improve visibility into the review process. We're adding a new event to VSA — Merge request last approved at — which creates a stage that breaks down MR review steps even further for granular visibility. The power of data-driven decisions By leveraging GitLab’s VSA, we didn’t just identify bottlenecks – we gained actionable insights that led to measurable improvements in MR review time and overall developer productivity. We optimized merge request review cycles and increased developer throughput, validating our commitment to continuous improvement through measurement. Want to learn more about how VSA can help your team? Start a free, 60-day trial of GitLab Ultimate, customize your value streams, and see how you can make improvements throughout the SDLC for your teams. Read more Optimize value stream efficiency to do more with less, faster New Scheduled Reports Generation tool simplifies value stream management Value stream analytics documentation Value stream management: Total Time Chart simplifies top-down optimization flow
As organizations grow, managing internal packages becomes increasingly complex. While traditional package managers, like JFrog Artifactory and Sonatype Nexus, use a centralized repository approach, GitLab takes a different path that aligns with modern development teams' work. In this post, we'll explore how to effectively structure your GitLab Package Registry for enterprise scale, focusing on Maven and npm packages as examples. Understanding the GitLab Package Registry model If you're coming from a traditional package manager, GitLab's approach might initially seem different. Instead of a single centralized repository, GitLab integrates package management directly into your existing project and group structure. This means: Teams publish packages to specific projects where the code lives Teams consume packages from root group registries that aggregate all packages below them Access control inherits from your existing GitLab permissions This model offers several advantages: Clear ownership of packages alongside their source code Granular access control without additional configuration Simplified CI/CD integration Natural alignment with team structures Single URL for accessing all company packages through root group consumption The power of root group package registry While GitLab supports package consumption at various group levels, using the root group level has emerged as a best practice among our users. Here's why: Single access point: One URL provides access to all private packages across your organization Consistent package naming: Group-level endpoints allow teams to maintain their preferred naming conventions without conflicts Simplified configuration: All developers can use the same configuration to access packages Secure access management: Combines with deploy tokens for easy rotation and access control Hierarchical organization: Naturally maps to your organizational structure while maintaining unified access Real-world example: Enterprise structure Let's look at how this works in practice with a large enterprise: company/ (root group) ├── retail-division/ │ ├── shared-libraries/ # Division-specific shared code │ └── teams/ │ ├── checkout/ # Team publishes packages here │ └── inventory/ # Team publishes packages here ├── banking-division/ │ ├── shared-libraries/ # Division-specific shared code │ └── teams/ │ ├── payments/ # Team publishes packages here │ └── fraud/ # Team publishes packages here └── shared-platform/ # Enterprise-wide shared code ├── java-commons/ # Shared Java libraries └── ui-components/ # Shared UI components Publishing configuration Teams publish packages to their specific project registries, maintaining clear ownership: Maven example gitlab-maven ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/maven npm example // ui-components/package.json { "name": "@company/ui-components", "publishConfig": { "registry": "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/npm/" } } Consuming configuration The power of root group consumption comes into play here. All teams configure a single endpoint for package access: Maven example gitlab-maven https://gitlab.example.com/api/v4/groups/company/-/packages/maven npm example # Any project's .npmrc @company:registry=https://gitlab.example.com/api/v4/groups/company/-/packages/npm/ This configuration automatically provides access to all packages across your organization while maintaining the benefits of project-based publishing. Authentication and access control GitLab's model simplifies authentication through deploy tokens and CI/CD integration. For CI/CD pipelines GitLab automatically handles authentication in pipelines using CI_JOB_TOKEN: # .gitlab-ci.yml publish: script: - mvn deploy # or npm publish # CI_JOB_TOKEN provides automatic authentication For development Use group deploy tokens for package consumption: Create read-only deploy tokens at the root group level Rotate tokens periodically for security Share a single configuration across all developers Benefits of root group package registry Simplified configuration One URL for all package access Consistent setup across teams Easy token rotation Clear ownership Packages stay with their source code Teams maintain control over publishing Version history tied to project activity Natural organization Matches your company structure Supports team autonomy Enables cross-team collaboration Getting started Set up your root group Create a clear group structure Configure appropriate access controls Create group deploy tokens Configure team projects Set up project-level publishing Implement CI/CD pipelines Document package naming conventions Standardize consumption Configure root group registry access Share deploy tokens securely Document package discovery process Summary GitLab's package registry model, particularly when leveraging root group consumption, offers a powerful solution for enterprise package management. By combining project-based publishing with root group consumption, organizations get the best of both worlds: clear ownership and simplified access. This approach scales naturally with your organization while maintaining security and ease of use. Start by implementing this model with a single team or division, and expand as you see the benefits of this integrated approach. Remember that while this post focused on Maven and npm, the same principles apply to all package types supported by GitLab. Get started with package registries today! Sign up for a free, 60-day trial of GitLab Ultimate.
Is GitOps still GitOps if you are not using a git repository as your deployment artifact? While git remains central to GitOps workflows, storing infrastructure definitions as Open Container Initiative (OCI) artifacts in container registries has seen a rise in adoption as the source for GitOps deployments. In this article, we will dive deeper into the ideas behind this trend and how GitLab features support this enhancement to GitOps workflows. What is GitOps? The OpenGitOps project has defined four principles for the practice of GitOps: A system managed by GitOps must have its desired state expressed declaratively. Desired state is stored in a way that enforces immutability and versioning, and retains a complete version history. Software agents automatically pull the desired state declarations from the source. Software agents continuously observe actual system state and attempt to apply the desired state. An example of GitOps is storing the Kubernetes manifests for a microservice in a GitLab project. Those Kubernetes resources are then continuously reconciled by a controller running on the Kubernetes cluster where the microservice is deployed to. This allows engineers to manage infrastructure using the same workflows as working with regular code, such as opening merge requests to make and review changes and versioning changes. GitOps also has operational benefits such as preventing configuration drift and helps engineers audit what changes led to certain outcomes with deployments. Benefits and limitations of git in GitOps workflows While git is an essential piece of GitOps workflows, git repositories were not designed to be deployed by GitOps controllers. Git does provide the ability for engineers to collaborate on infrastructure changes and audit these changes later on, but controllers do not need to download an entire git repository for a successful deployment. GitOps controllers simply need the infrastructure defined for a particular environment. Additionally, an important piece of the deployment process is to sign and verify deployments to assure deployment changes to an environment are coming from a trusted source. While git commits can be signed and verified by GitOps controllers, commits may also capture other details not related to the deployment itself (e.g., documentation changes, updates to other environments, and git repository restructuring) or not enough of the deployment picture as a deployment may consist of multiple commits. This again feels like a case this git feature wasn’t designed for. Another challenging aspect of git in GitOps workflows is that it can sometimes lead to more automation than expected. Soon after merging a change to the watched branch, it will be deployed. There are no controls in the process outside of git. How can you make sure that nothing gets deployed on a Friday late afternoon? What if teams responsible for deployment do not have permissions to merge changes in certain GitLab projects? Using OCI images adds a pipeline into the process, including all the delivery control features, like approvals or deploy freezes. OCI images The Open Container Initiative has helped to define standards around container formats. While most engineers are familiar with building Dockerfiles into container images, many may not be as familiar with storing Kubernetes manifests in a container registry. Because GitLab’s Container Registry is OCI compliant, it allows for users to push Kubernetes manifests for a particular environment to a container registry. GitOps controllers, such as Flux CD, can use the manifests stored in this OCI artifact instead of needing to clone an entire git repository. Often in GitOps workflows, a git repository can include the infrastructure definitions for all environments that a microservice will be deployed to. By packaging the Kubernetes manifests for only a specific environment, Flux CD can download the minimum files needed to carry out a deployment to a specific environment. Security benefits of using OCI artifacts As mentioned previously, signing and verifying the artifacts to be deployed to an environment adds an additional layer of security for software projects. After Kubernetes manifests are pushed to a container registry, a tool like Sigstore Cosign can be used to sign the OCI image with a private key that can be securely stored in a GitLab project as a CI/CD variable. Flux CD can then use a public key stored on a Kubernetes cluster to verify that a deployment is coming from a trusted source. Using GitLab to push and sign OCI images GitLab offers many features that help simplify the process of packaging, signing, and deploying OCI images. A common way to structure GitLab projects with GitOps workflows is to have separate GitLab projects for microservices’ code and a single infrastructure repository for all microservices. If an application is composed of n microservices, this would require having n +1 GitLab projects for an application. The artifact produced by a code project is usually a container image that will be used to package the application. The infrastructure or delivery project will contain the Kubernetes manifests defining all the resources required to scale and serve traffic to each microservice. The artifact produced by this project is usually an OCI image used to deploy the application and other manifests to Kubernetes. In this setup, separation of environments is handled by defining Kubernetes manifests in separate folders. These folders represent environments (e.g., development, staging, and production) that will host the application. When changes are made to the code project and a new container image is pushed, all that needs to be done to deploy these changes via GitLab’s integration with Flux CD is to edit the manifests under the environment folder to include the new image reference and open a merge request. Once that merge request is reviewed, approved, and merged, the delivery project’s CI/CD job will push a new OCI image that Flux CD will pick up and deploy to the new environment. Signing an OCI image is as simple as including Cosign in your project’s CI/CD job. You can simply generate a new public and private key with Cosign by running the commands below locally. Just make sure to log in to your GitLab instance with the glab CLI and replace the [PROJECT_ID] for the Cosign command with your delivery project’s ID. glab auth login cosign generate-key-pair gitlab://[PROJECT_ID] Once the cosign command runs successfully, you can see the Cosign keys added to your project under the CI/CD variables section under the key names COSIGN_PUBLIC_KEY and COSIGN_PRIVATE_KEY. Example CI/CD job A GitLab CI/CD job for pushing an OCI image will look something like the following: frontend-deploy: rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH changes: paths: - manifests/dev/frontend-dev.yaml trigger: include: - component: gitlab.com/components/fluxcd/oci-artifact@0.3.1 inputs: version: 0.3.1 kubernetes_agent_reference: gitlab-da/projects/tanuki-bank/flux-config:dev registry_image_url: "oci://$CI_REGISTRY_IMAGE/frontend" image_tag: dev manifest_path: ./manifests/dev/frontend-dev.yaml flux_oci_repo_name: frontend flux_oci_namespace_name: frontend-dev signing_private_key: "$COSIGN_PRIVATE_KEY" The GitLab CI/CD Catalog offers a GitLab-maintained CI/CD component for working with OCI artifacts and Flux CD. This component allows development teams to push Kubernetes manifests as OCI images to GitLab’s Container Registry or an external container registry, sign the OCI image using Cosign, and immediately reconcile the newly pushed image via Flux CD. In the example above, the Flux CD component is included in a .gitlab-ci.yml file of a GitLab project. Using the component’s inputs, users can define what registry to push the image to (i.e., registry_image_url and image tag), the file path to Kubernetes manifests that will be pushed (i.e., manifest_path), the cosign private key used to sign images (i.e., signing_private_key), and the Kubernetes namespace and Flux CD OCIRepository name needed to sync updates to an environment (i.e., flux_oci_namespace_name and flux_oci_repo_name). The kubernetes_agent_reference allows GitLab CI/CD jobs to inherit the kubeconfig needed to access a Kubernetes cluster without needing to store a kubeconfig CI/CD variable in each GitLab project. By setting up the GitLab agent for Kubernetes, you can configure all GitLab projects’ CI/CD jobs in a GitLab group to inherit permissions to deploy to the Kubernetes cluster. The agent for Kubernetes context is typically configured wherever you configure the GitLab Agent for Kubernetes in your GitLab group. It is typically recommended that this be done in the project where Flux CD is managed. More information on configuring the agent for CI/CD access can be found in our CI/CD workflow documentation. The variables $COSIGN_PRIVATE_KEY, $FLUX_OCI_REPO_NAME, and $FRONTEND_DEV_NAMESPACE are values stored as CI/CD variables to easily access and mask these sensitive pieces of data in CI/CD logs. The $CI_REGISTRY_IMAGE is a variable that GitLab jobs have available by default that specifies the GitLab project’s container registry. Deploy OCI images Using Flux CD with your GitLab projects, you can automate deployments and signing verification for your microservice’s environments. Once Flux CD is configured to sync from a GitLab project, you could add the following Kubernetes custom resource definitions to your project to sync your pushed OCI image. apiVersion: v1 kind: Namespace metadata: name: frontend-dev labels: name: frontend-dev --- apiVersion: bitnami.com/v1alpha1 kind: SealedSecret metadata: name: cosign-public-key namespace: frontend-dev spec: encryptedData: cosign.pub: AgAKgLf4VbVzJOmr6++k81LlFayx88AELaUQFNOaXmBF4G+fBfBYeABl0skNvMAa1UrPVNSfMIHgFoYHoO96g576a+epk6V6glOI+++XvYbfsygof3GGxe0nL5Qh2b3ge0fNpyd0kTPSjTj0YUhRhKtMGMRSRw1jrwhNcGxCHK+Byibs52v8Np49KsIkeZKbzLdgYABkrv+k0j7hQM+jR180NpG+2UiRvaXpPuogxkbj61FEqWGrJHk8IVyfl3eh+YhoXxOHGDqko6SUC+bUZPDBlU6yKegO0/8Zq3hwulrSEsEjzRZNK+RFVMOLWWuC6h+WGpYhAMcsZPwjjJ/y29KLNa/YeqkN/cdk488QyEFc6ehCxzhH67HxIn2PDa+KkEOTv2TuycGF+Q00jKIizXF+IwLx/oRb3pTCF0AoAY8D8N3Ey+KfkOjsBON7gGID8GbQiJqX2IgIZxFMk0JRzxbRKOEqn+guLd5Shj7CD1a1Mkk0DxBdbqrGv2XNYUaFPI7xd3rZXUJZlnv+fsmwswsiGWRuXwim45HScWzQnfgLAe7tv3spVEGeaO5apl6d89uN21PBQnfE/zyugB//7ZW9tSp6+CSMyc5HynxI8diafqiwKPgvzLmVWRnkvxJijoXicRr3sCo5RudZPSlnjfd7CKdhwEVvLl7dRR4e/XBMdxCzk1p52Pl+3/kJR+LJii5+iwOpYrpVltSZdzc/3qRd19yMpc9PWpXYi7HxTb24EOQ25i21eDJY1ceplDN6bRtop2quzkjlwVeE2i4cEsX/YG8QBtQbop/3fjiAjKaED3QH3Ul0PECS9ARTScSkcOL3I00Xpp8DyD+xH0/i9wCBRDmH3yKX18C8VrMq02ALSnlP7WCVVjCPzubqKx2LPZRxK9EG0fylwv/vWQzTUUwfbPQZsd4c75bSTsTvxqp/UcFaXA== template: metadata: name: cosign-public-key namespace: frontend-dev --- apiVersion: source.toolkit.fluxcd.io/v1beta2 kind: OCIRepository metadata: name: frontend namespace: frontend-dev spec: interval: 1m url: oci://registry.gitlab.com/gitlab-da/projects/tanuki-bank/tanuki-bank-delivery/frontend ref: tag: dev verify: provider: cosign secretRef: name: cosign-public-key --- apiVersion: kustomize.toolkit.fluxcd.io/v1 kind: Kustomization metadata: name: frontend namespace: frontend-dev spec: interval: 1m targetNamespace: frontend-dev path: "." sourceRef: kind: OCIRepository name: frontend prune: true The Kustomization resource allows for further customization of Kubernetes manifests and also specifies which namespace to deploy resources to. The OCIRepository resource for Flux CD allows users to specify the OCI image repository reference and tag to regularly sync from. Additionally, you will notice the verify.provider and verify.secretRef properties. These fields allow you to verify that the OCI image deployed to the cluster was signed by the corresponding Cosign private key used in the earlier CI/CD job. The public key needs to be stored in a Kubernetes secret that will need to be present in the same namespace as the OCIRepository resource. To have this secret managed by Flux CD and not store the secret in plain text, you can consider using SealedSecrets to encrypt the value and have it be decrypted cluster side by a controller. For a simpler approach not requiring SealedSecrets, you can deploy the secret via a GitLab CI/CD job using the kubectl CLI. In the non-sealed secret approach, you would simply remove the SealedSecret included above and run the job to deploy the public key secret before running the job to push the new OCI image. This will make sure the secret is stored securely in GitLab and make sure the secret can be accessed on the cluster by the OCIRepository. While this approach is a bit simpler, just note this is not a suitable approach for managing secrets in production. The benefits of OCI, GitLab, and GitOps OCI artifacts allow for GitOps teams to take deployments even further with added security benefits and allowing for deployments to be minimal. Users still gain all the benefits offered by git as far as having a source of truth for infrastructure and collaborating on projects. OCI images add a packaging approach that improves the deployment aspect of GitOps. GitLab continues to learn from our customers and the cloud native community on building experiences that help simplify GitOps workflows. To get started using some of the features mentioned in this blog, you can sign up for a 60-day free trial of GitLab Ultimate. We would also love to hear from users about their experiences with these tools, and you can provide feedback in the community forum.
The certificate-based Kubernetes integration was deprecated in GitLab November 2021, and is available on GitLab.com only to previous users. In May 2025, the integration will sunset on GitLab.com and will stop working. Customers often use the integration to deploy applications to production and non-production environments. As a result, failure to migrate to other options could cause a critical incident in your application delivery pipelines. This post outlines the alternative features that GitLab offers, points out how you can identify the potential impact on your GitLab.com groups and projects, and offers links to the GitLab documentation to learn more about the necessary migration steps. Recommended alternative: The GitLab agent for Kubernetes The GitLab agent for Kubernetes represents a significant advancement over the certificate-based integration, offering enhanced security, reliability, and functionality. Here are the key benefits of migrating to the agent-based approach: Enhanced security Eliminates the need for storing cluster credentials in GitLab Provides secure, bidirectional communication between GitLab and your clusters Supports fine-grained access control and authorization policies Enables secure GitOps workflows with pull-based deployments Improved reliability Maintains persistent connections, reducing deployment failures Handles network interruptions gracefully Provides better logging and troubleshooting capabilities Supports automatic reconnection and state recovery Advanced features Real-time cluster information integrated into the GitLab UI Integration with GitLab CI/CD pipelines Support for multiple clusters and multitenant environments Enhanced GitOps capabilities by integrating with FluxCD Get started with the GitLab agent for Kubernetes If you haven't tried the GitLab Agent for Kubernetes yet, we strongly recommend going through the getting started guides. These guides will walk you through the basic setup and help you understand how the agent works in your environment. The hands-on experience will help make the migration process smoother. Impact assessment We implemented a dedicated API endpoint to query all the certificate-based clusters within a GitLab group hierarchy. We recommend starting with this API to see if you have any clusters that need to be migrated. Once you identify the clusters, you should: Find group and project owners using the certificate-based integration. Check CI/CD pipelines for direct Kubernetes API calls. Identify Auto DevOps projects using the old integration. List any GitLab-managed clusters in use. Set up the agent in the affected clusters. Follow the guidance provided in this post and record your progress in a tracking issue. Update your CI/CD integration The legacy certificate-based integration works using GitLab CI/CD. Because the agent seamlessly integrates with GitLab CI/CD pipelines, you can use it to replace the certificate-based integration with relatively little effort. The agent-based CI/CD integration offers several improvements over the certificate-based approach: Direct cluster access: CI/CD jobs can interact with clusters through the agent without requiring separate credentials. Enhanced security: You don't need to store cluster credentials in CI/CD variables. Simplified configuration: A single agent configuration file manages all cluster interactions. Better performance: Persistent connections reduce deployment overhead. Flexible authorization: On GitLab Premium and Ultimate, you can rely on impersonation features to restrict CI/CD jobs in the cluster. At a high level, there are three steps to migrating your existing CI/CD pipelines: Set up the agent by following the getting started guides. Share the agent connection with the necessary groups and projects.. Select the agent in the pipeline jobs. You can read more about migrating Kubernetes deployments in general or about the agent CI/CD integration in the documentation. Migrate your Auto DevOps configuration Auto DevOps is a set of CI/CD templates that are often customized by users. With Auto DevOps, you can automatically configure your CI/CD pipelines to build, test, and deploy your applications based on best practices. It's commonly used with the certificate-based integration for deploying applications to Kubernetes clusters. If you use Auto DevOps and you rely on the certificate-based integration, you need to transition to the agent-based deployment mechanism. The migration process is straightforward: Set up the CI/CD integration as described above. Configure the KUBE_CONTEXT environment variable to select an agent. Remove the old certificate-based cluster integration. You can read more about using Auto DevOps with the agent for Kubernetes in the documentation. Transition from GitLab-managed clusters to GitLab-managed Kubernetes resources With GitLab-managed clusters, GitLab automatically creates and manages Kubernetes resources for your projects. When you allow GitLab to manage your cluster, it creates RBAC resources like a Namespace and ServiceAccount. If you use GitLab-managed clusters, you should transition to GitLab-managed Kubernetes resources, which offers a more flexible and secure approach to cluster management. To migrate: Document your existing cluster configuration. Create corresponding Kubernetes resource definitions. Store configurations in your repository. Configure the GitLab agent to manage these resources. Verify resource management and deployment. Remove the old cluster integration. You can read more about GitLab-managed Kubernetes resources in the documentation. Manage cloud provider clusters created through GitLab If you created Kubernetes clusters through the GitLab integration with Google Kubernetes Engine (GKE) or Amazon Elastic Kubernetes Service (EKS), these clusters were provisioned in your respective cloud provider accounts. After the certificate-based integration is removed: Your clusters will remain fully operational in Google Cloud or AWS. You will need to manage these clusters directly through your cloud provider's console: GKE clusters through Google Cloud Console EKS clusters through AWS Management Console To view cluster information within GitLab: Install the GitLab agent for Kubernetes. Configure the Kubernetes dashboard integration. Check the dashboard for cluster details and resource information. This change only affects how you interact with the clusters through GitLab – it does not impact the clusters' operation or availability in your cloud provider accounts. You should still migrate your deployment setups as described above. What should I do next? To minimize the impact to you and your infrastructure, you should follow these steps: Check if you are impacted as soon as possible. Plan your migration timeline before May 2025. Start with non-production environments to gain experience. Document your current setup and desired state. Test the agent-based approach in a staging environment. Gradually migrate production workloads. Monitor and validate the new setup. The migration to the GitLab agent for Kubernetes represents a significant improvement in how GitLab interacts with Kubernetes clusters. While the migration requires careful planning and execution, the benefits in terms of security, reliability, and functionality make it a worthwhile investment for your DevSecOps infrastructure.
"We need to migrate hundreds of container images from Amazon Elastic Container Registry (ECR) to GitLab. Can you help?" This question kept coming up in conversations with platform engineers. They were modernizing their DevSecOps toolchain with GitLab but got stuck when faced with moving their container images. While each image transfer is simple, the sheer volume made it daunting. One platform engineer perfectly said, "I know exactly what needs to be done – pull, retag, push. But I have 200 microservices, each with multiple tags. I can't justify spending weeks on this migration when I have critical infrastructure work." The challenge That conversation sparked an idea. What if we could automate the entire process? When platform teams move their CI/CD to GitLab, migrating container images shouldn't be the bottleneck. The manual process is straightforward but repetitive – pull each image, retag it, and push it to GitLab's Container Registry. Multiply this by dozens of repositories and multiple tags per image, and you're looking at days or weeks of tedious work. The solution We set out to create a GitLab pipeline that would automatically do all this heavy lifting. The goal was simple: Give platform engineers a tool they could set up in minutes and let run overnight, waking up to find all their images migrated successfully. Setting up access First things first – security. We wanted to ensure teams could run this migration with minimal AWS permissions. Here's the read-only identity and access management (IAM) policy you'll need: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ecr:GetAuthorizationToken", "ecr:BatchCheckLayerAvailability", "ecr:GetDownloadUrlForLayer", "ecr:DescribeRepositories", "ecr:ListImages", "ecr:DescribeImages", "ecr:BatchGetImage" ], "Resource": "*" } ] } GitLab configuration With security handled, the next step is setting up GitLab. We kept this minimal - you'll need to configure these variables in your CI/CD settings: AWS_ACCOUNT_ID: Your AWS account number AWS_DEFAULT_REGION: Your ECR region AWS_ACCESS_KEY_ID: [Masked] AWS_SECRET_ACCESS_KEY: [Masked] BULK_MIGRATE: true The migration pipeline Now for the interesting part. We built the pipeline using Docker-in-Docker to handle all the image operations reliably: image: docker:20.10 services: - docker:20.10-dind before_script: - apk add --no-cache aws-cli jq - aws sts get-caller-identity - aws ecr get-login-password | docker login --username AWS --password-stdin - docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} ${CI_REGISTRY} The pipeline works in three phases, each building on the last: Discovery First, it finds all your repositories: REPOS=$(aws ecr describe-repositories --query 'repositories[*].repositoryName' --output text) Tag enumeration Then, for each repository, it gets all the tags: TAGS=$(aws ecr describe-images --repository-name $repo --query 'imageDetails[*].imageTags[]' --output text) Transfer Finally, it handles the actual migration: docker pull ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${repo}:${tag} docker tag ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${repo}:${tag} ${CI_REGISTRY_IMAGE}/${repo}:${tag} docker push ${CI_REGISTRY_IMAGE}/${repo}:${tag} What you get Remember that platform engineer who didn't want to spend weeks on migration? Here's what this solution delivers: automated discovery and migration of all repositories and tags consistent image naming between ECR and GitLab error handling for failed transfers clear logging for tracking progress Instead of writing scripts and babysitting the migration, the platform engineer could focus on more valuable work. Usage Getting started is straightforward: Copy the .gitlab-ci.yml to your repository. Configure the AWS and GitLab variables. Set BULK_MIGRATE to "true" to start the migration. Best practices Through helping teams with their migrations, we've learned a few things: Run during off-peak hours to minimize the impact on your team. Keep an eye on the pipeline logs - they'll tell you if anything needs attention. Don't decommission ECR until you've verified all images transferred successfully. For very large migrations, consider adding rate limiting to avoid overwhelming your network We've open-sourced this pipeline in our public GitLab repository because we believe platform engineers should spend time building valuable infrastructure, not copying container images. Feel free to adapt it for your needs or ask questions about implementation. Get started with this and other package components with our CI/CD Catalog documentation.
Welcome to our "Getting started with GitLab" series, where we help newcomers get familiar with the GitLab DevSecOps platform. GitLab is much more than just a place to store your code. It is an AI-powered DevSecOps platform with tools to help you plan, organize, track, and successfully deliver your projects. This post will guide you through GitLab's key project management features and show you how to leverage them effectively. Why GitLab for project management? Imagine having your code repository, issue tracker, and communication platform all seamlessly integrated in one place. That's the power of GitLab. By centralizing everything, you can streamline your workflow, enhance collaboration, and keep your projects moving forward. No more jumping between different tools and losing track of information. GitLab brings it all together, making it easier to manage your projects from start to finish. Key components of GitLab project management Let's break down the essential elements: Epics: Think of epics as the big picture. They represent major features, overarching goals, or long-term initiatives within your project. Need to revamp your website? That's an epic! Epics help you organize your work into larger, manageable chunks. Issues: Issues are the individual tasks or work items that contribute to your project goals. Each issue represents a specific action, like "design the homepage" or "write the 'about us' page." Issues are the building blocks of your project, and they provide a clear way to track individual tasks. Labels: Labels are like tags that help you categorize and filter your work. You can use labels to indicate priority (e.g., high, medium, low), status (e.g., to do, in progress, done), or assign issues to specific teams or individuals. Labels provide a flexible way to organize and prioritize your work. Boards: GitLab's issue boards are your visual workspace. They provide a Kanban-style view of your project, allowing you to see the status of all your issues at a glance. Drag and drop issues across different lists (e.g., "To Do," "Doing," "Done") to visualize your workflow and track progress. In GitLab, you can create boards for issues and Epics. Milestones: Milestones mark significant checkpoints or target dates within your project. They help you track progress towards specific goals and deadlines. For example, you might have milestones for completing a major feature, releasing a beta version, or launching the final product. Tasks: For those extra granular steps, break down your issues into smaller tasks. This helps with delegation, clarifies individual responsibilities, and ensures nothing gets overlooked. Tasks provide a way to create checklists within issues, making it easier to track progress on complex tasks. Deep dive into the features 1. Epics: The big picture Creating epics: Navigate to your group's "Epics" menu under “Plan”. Click New epic and give it a descriptive title and a clear description outlining the goal. You can also specify the start and end date of the epic – this is useful when using Roadmaps. Roadmaps: Add your epics to a roadmap to visualize your project timeline and long-term goals. Roadmaps provide a bird's-eye view of your project plan, making it easy to see the big picture and track progress towards major milestones. 2. Issues: Getting things done Creating issues: In your project, go to the "Issues" menu under “Plan” and click New issue. Provide a concise and descriptive title like "Design Homepage Wireframes," assign it to a team member, set a due date, and add a detailed description outlining the task's requirements. GitLab Duo: You can leverage the power of GitLab Duo to create detailed issue descriptions with just a little hint of what you want to achieve. Weighting: Estimate the effort required for each issue by assigning weights. This helps with planning and prioritization. For example, a simple task might have a weight of 1, while a more complex task might have a weight of 5. 3. Labels: Organizing your work Creating labels: Go to your project's "Issues" tab and click Labels. Create custom labels with clear names to categorize your issues. For example, create labels like Priority: High, Status: In Progress, or Team: Design. Apply these labels to your issues to keep them organized and easily filterable. 4. Boards: Visualizing your workflow Kanban boards: GitLab's boards provide a Kanban-style view of your project. Create lists like "To Do," "Doing," and "Done" to represent the stages of your workflow. Drag and drop issues across these lists to visualize their progress. Customizing boards: Tailor your boards to match your specific workflow. Add more columns, filter issues by labels or assignees, and set up swim lanes to categorize issues by epics or other criteria. 5. Tasks: Breaking down the work Creating tasks: Within an issue, use the checklist markdown syntax to create a task list. Each item in the list represents a smaller step within the larger issue. For example, in the issue "Design Homepage Wireframes," you might have tasks like "Sketch initial concepts," "Create digital wireframes," and "Get feedback from stakeholders." To create a Task, click on the Add button in the "Child Items" section of an issue’s page. Then, enter the title of the task, and click Create Task. 6. Milestones: Tracking progress Setting milestones: Define milestones to mark significant points in your project, like completing a specific feature or reaching a key deadline. Give your milestones clear titles and due dates. Associating with issues: Link issues and epics to milestones to track progress towards those goals. This helps you see how individual tasks contribute to the overall project plan. Creating a milestone: Under the "Plan" dropdown menu, click on Milestones > New milestone. Specify the milestone title, description, and start and due dates. 7. Iterations: Working in sprints Defining iterations: If you're using an Agile workflow, define iterations (sprints) with specific start and end dates. This helps you break down your work into smaller, more manageable time boxes. Assigning issues: Assign issues to iterations to plan your work in shorter cycles and focus on delivering incremental value. 8. Time tracking: Measuring effort Logging time: Within an issue, use the "/spend" quick action followed by the time spent (e.g., "/spend 2h 30m") to log your work. This helps you track the actual time spent on each task. Analyzing data: Generate time tracking reports to gain insights into project progress, team efficiency, and identify potential bottlenecks. 9. Dependencies: Managing workflow Linking issues: Create dependencies between issues to ensure tasks are completed in the correct order. For example, if issue A must be completed before issue B can begin, you can create a dependency between them. This helps you visualize the workflow and avoid potential roadblocks. 10. Templates: Streamlining issue creation Creating templates: Create issue templates to standardize the information captured for common tasks, saving you time and ensuring consistency. For example, you could create a template for bug reports that includes fields for steps to reproduce expected behavior and actual behavior. Collaboration is key GitLab fosters collaboration through the following: Comments: Discuss issues and epics directly within GitLab. Use comments to provide updates, ask questions, and share feedback. Mentions: Use @ to mention specific team members and notify them of updates or request their input. Discussions: Engage in threaded discussions within issues and epics to brainstorm ideas, solve problems together, and keep everyone informed. Ready to get started? Now that you've explored the power of GitLab's project management features, it's time to put them into practice! Create a sample project, experiment with different features, and discover how GitLab can transform your workflow. You can also learn more about how GitLab can help you facilitate Kanban and Scrum in the GitLab documentation. Don't have a GitLab account? Sign up for a free, 60-day trial of GitLab Ultimate and GitLab Duo today! Learn more Getting started with GitLab: How to manage users Getting started with GitLab: How to import your projects to GitLab
Secrets left exposed in outdated repositories pose significant risk for data breaches. For example, a still-active secret key can be exposed, leaving it vulnerable to exploitation. Secrets include access keys, API tokens, private keys, and other sensitive values. In this article, you'll learn how to use GitLab Secret Detection to scan a repository’s full commit history, including all branches, to detect sensitive secrets. In addition, you will discover how to view the results directly within the GitLab UI without the need for any integration. All it takes is just a couple of lines of code in your .gitlab-ci.yml pipeline file. Scan every corner of your repository We will use the sample repository shown in the screenshot below as an example. To keep things simple, there is only a README.md file present in the default branch of this repository. At first glance, it may seem like the repository is empty and that there are probably no sensitive secrets in this repository. But what we are looking at is only the state of the default branch, which is the main branch in this example. There could be feature branches in this repository created weeks, months, or years ago with sensitive secrets. It is also possible that a file with a secret was accidentally pushed to the repo and then deleted right after. However, it likely was not deleted correctly and is still in the commit history. We are going to enable GitLab Secret Detection scanner and set the SECRET_DETECTION_HISTORIC_SCAN variable to true so that the content of all branches in the repository is scanned. include: - template: Jobs/Secret-Detection.gitlab-ci.yml secret_detection: variables: SECRET_DETECTION_HISTORIC_SCAN: "true" By setting the SECRET_DETECTION_HISTORIC_SCAN variable to true, GitLab Secret Detection looks into every branch and commit of your repository. It ensures that no sensitive information — whether from a feature branch or an old commit — is left unchecked. Results of the scan Two sensitive secrets were identified in the repository. One is a password in a .env file that was deleted from the repository, but the commit containing it was not removed from the git history. The other is an AWS Access Token found in a feature branch. These exposed secrets could compromise the organization’s security. You can click on the AWS Access Token result to see more details, including the file location. You can also create a GitLab issue to triage the vulnerability with one click. If you’re using the Jira integration, you can create a Jira ticket directly from the vulnerability page as well. Why scanning for secrets matters Anyone with access to the repository can misuse the secret to gain unauthorized access to private resources and sensitive data. In addition to scanning a repository’s full commit history across all branches, GitLab Secret Detection also helps you take a multilayered approach to detecting secrets: Secret push protection - scans commits for secrets during a push and blocks it if secrets are detected, unless skipped, reducing the risk of leaks. Pipeline secret detection - scans files after they’ve been committed and pushed to a GitLab repository. Client-side secret detection - scans comments and descriptions in issues and merge requests for secrets before they're saved to GitLab. * Automatic response to leaked secrets - automatically revokes certain types of leaked secrets and notifies the partner that issued the secret. You can adjust pipeline secret detection to suit your needs by modifying, extending, or replacing the default ruleset. For instance, you can define custom rules using regex patterns to detect sensitive data like credit card numbers, phone numbers, or other information specific to your organization. Try GitLab Secret Detection Enable Secret Detection in your GitLab pipeline. Set SECRET_DETECTION_HISTORIC_SCAN: true. Push and trigger a pipeline to scan all branches and commits. GitLab makes securing your code simple and comprehensive. Don’t let an old branch or commit compromise your security — give historical scans a try today! Sign up for a free 60-day trial of GitLab Ultimate to get started with security scanners like Secret Detection.
Air-gapped environments are computer networks or systems that are physically isolated from unsecured networks, such as the public internet or unsecured local area networks. This isolation is implemented as a security measure to protect sensitive data and critical systems from external cyber threats by providing: Enhanced security: By physically isolating systems from external networks, air-gapped environments help prevent remote attacks, malware infections, and unauthorized data access. This is crucial for highly sensitive data and critical systems. Data protection: Air-gapping provides the strongest protection against data exfiltration since there's no direct connection that attackers could use to steal information. Critical infrastructure protection: For systems that control vital infrastructure (like power plants, water treatment facilities, or military systems), air-gapping helps prevent potentially catastrophic cyber attacks. Compliance requirements: Many regulatory frameworks require air-gapping for certain types of sensitive data or critical systems, particularly in government, healthcare, and financial sectors. Malware protection: Without network connectivity, systems are protected from network-based malware infections and ransomware attacks. Even though air-gapped systems are isolated, they can still have vulnerabilities. Regular security scanning helps identify these weaknesses before they can be exploited. In this article, you will learn the different security scanners GitLab provides and how they can be added/updated in a limited-connectivity environment. GitLab security scanners in air-gapped environments GitLab provides a variety of different security scanners for the complete application lifecycle. The scanners that support air-gapped environments include: Static Application Security Testing (SAST) Dynamic Application Security Testing (DAST) Secret Detection Container Scanning Dependency Scanning API Fuzzing License Scanning By default, GitLab Self-Managed instances pull security scanner images from the public GitLab container registry (registry.gitlab.com) and store them within the built-in local GitLab container registry. I will demonstrate this flow below by running the following pipeline that scans for secrets on a sample project: include: - template: Jobs/Secret-Detection.gitlab-ci.yml When running the job in an internet-connected GitLab instance the job passes: GitLab Runner with internet access successfully pulling from external registry However, If I disable internet access to the VM running GitLab, the secret-detection job will fail to download the container image, causing the job to fail: GitLab Runner without internet access failing to pull from external registry Alternatively, if I set my GitLab Runners’ pull image policy to if-not-present from always, I can load the cached version of the scanner if it was run before on the internet by using the image stored in our local docker: GitLab Runner without internet access successfully pulling from internal registry cache Setting up offline scanning prerequisites Running these security scanners in an air-gapped environment requires the following: GitLab Ultimate subscription Offline cloud license GitLab Self-Managed cluster You can follow along with this tutorial in any GitLab Self-Managed EE instance (even those that are not air-gapped) to learn how to transfer and run images in an air-gapped environment. In this tutorial, I will demonstrate how to load scanner images onto a GitLab-EE instance running in a Google Compute VM where I cut off the EGRESS to everything by implementing firewall rules: # egress firewall rule to block all outbound traffic to the internet $ gcloud compute firewall-rules create deny-internet-egress \ --direction=EGRESS \ --priority=1000 \ --network=default \ --action=DENY \ --rules=all \ --destination-ranges=0.0.0.0/0 \ --target-tags=no-internet # Create an allow rule for internal traffic with higher priority $ gcloud compute firewall-rules create allow-internal-egress \ --direction=EGRESS \ --priority=900 \ --network=default \ --action=ALLOW \ --rules=all \ --destination-ranges=10.0.0.0/8,192.168.0.0/16,172.16.0.0/12 \ --target-tags=no-internet # Apply tag to VM $ gcloud compute instances add-tags YOUR_VM_NAME \ --zone=YOUR_ZONE \ --tags=no-internet Then, once I SSH into my VM, you can see we cannot connect to registry.gitlab.com: # showing I can’t access the gitlab container registry $ ping registry.gitlab.com PING registry.gitlab.com (35.227.35.254) 56(84) bytes of data. ^C --- registry.gitlab.com ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2031ms Note: I am still allowing ingress so I can copy files and SSH into the machine. Load security scanners in air-gapped environments To use the various security scanners on air-gapped environments, the GitLab Runner must be able to fetch the scanner container images from GitLab’s built-in container registry. This means that the container images for the security scanners must be downloaded and packaged in a separate environment with access to the public internet. The process of loading security scanners onto an air-gapped environment includes the following: Download and package container images from the public internet. Transfer images to offline environment. Load transferred images into offline container registry. Now let’s go over how we can implement GitLab Secret Detection in an air-gapped environment. Download and package container images from public internet Let’s download the container image for secret detection and store it within our local container registry. Other scanner images can be found in the offline deployments documentation. I will be using Podman desktop to download these images, but you can use Docker desktop or other alternatives. Pull the GitLab Secret Detection image. $ podman pull registry.gitlab.com/security-products/secrets:6 Trying to pull registry.gitlab.com/security-products/secrets:6... Getting image source signatures Copying blob sha256:999745130ac045f2b1c29ecce088b43fc4a95bbb82b7960fb7b8abe0e3801bf8 Copying blob sha256:a4f7c013bb259c146cd8455b7c3943df7ed84b157e42a2348eef16546d8179b1 Copying blob sha256:1f3e46996e2966e4faa5846e56e76e3748b7315e2ded61476c24403d592134f0 Copying blob sha256:400a41f248eb3c870bd2b07073632c49f1e164c8efad56ea3b24098a657ec625 Copying blob sha256:9090f17a5a1bb80bcc6f393b0715210568dd0a7749286e3334a1a08fb32d34e6 Copying blob sha256:c7569783959081164164780f6c1b0bbe1271ee8d291d3e07b2749ae741621ea3 Copying blob sha256:20c7ca6108f808ad5905f6db4f7e3c02b21b69abdea8b45abfa34c0a2ba8bdb5 Copying blob sha256:e8645a00be64d77c6ff301593ce34cd8c17ffb2b36252ca0f2588009a7918d2e Copying config sha256:0235ed43fc7fb2852c76e2d6196601968ae0375c72a517bef714cd712600f894 Writing manifest to image destination WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64) 0235ed43fc7fb2852c76e2d6196601968ae0375c72a517bef714cd712600f894 $ podman images REPOSITORY TAG IMAGE ID CREATED SIZE registry.gitlab.com/security-products/secrets 6 0235ed43fc7f 4 hours ago 85.3 MB Save the image as a tarball. $ podman save -o secret-detection.tar registry.gitlab.com/security-products/secrets:6 $ chmod +r secret-detection.tar $ ls -al secret-detection.tar -rw-r--r--@ 1 fern staff 85324800 Jan 10 10:25 secret-detection.tar Alternatively, you can use the official GitLab template on an environment with internet access to download the container images needed for the security scanners and save them as job artifacts or push them to the container registry of the project where the pipeline is executed. Transfer images to offline environment Next, let's transfer the tarball to our air-gapped environment. This can be done in several ways, depending on your needs, such as: Physical media transfer Data diodes Guard systems Cross-domain solutions (CDS) I will SCP (Secure Copy Protocol) the tarball directly to my VM that does not have egress access, but does allow ingress. As this is just for demonstration purposes, make sure to consult your organization's security policies and transfer procedures for air-gapped environments. Verify the image is not cached Before transferring the file, I’ll delete the Docker images on my GitLab instance pertaining to secret detection to make sure they aren't cached: $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.gitlab.com/security-products/secrets 6 0235ed43fc7f 9 hours ago 84.8MB registry.gitlab.com/security-products/secrets 16d88433af61 17 hours ago 74.9MB $ docker image rmi 16d88433af61 -f Untagged: registry.gitlab.com/security-products/secrets@sha256:f331da6631d791fcd58d3f23d868475a520f50b02d64000e2faf1def66c75d48 Deleted: sha256:16d88433af618f0b405945031de39fe40b3e8ef1bddb91ca036de0f5b32399d7 Deleted: sha256:1bb06f72f06810e95a70039e797481736e492201f51a03b02d27db055248ab6f Deleted: sha256:a5ef2325ce4be9b39993ce301f8ed7aad1c854d7ee66f26a56a96967c6606510 Deleted: sha256:f7cdac818a36d6c023763b76a6589c0db7609ca883306af4f38b819e62f29471 Deleted: sha256:5eabf4d47287dee9887b9692d55c8b5f848b50b3b7248f67913036014e74a0e9 Deleted: sha256:51b7cb600604c0737356f17bc02c22bac3a63697f0bf95ba7bacb5b421fdb7da Deleted: sha256:1546193b011d192aa769a15d3fdd55eb4e187f201f5ff7506243abb02525dc06 Deleted: sha256:1ea72408d0484c3059cc0008539e6f494dc829caa1a97d156795687d42d9cb57 Deleted: sha256:1313ee9da7716d85f63cfdd1129f715e9bbb6c9c0306e4708ee73672b3e40f26 Deleted: sha256:954ebfd83406f0dfed93eb5157ba841af5426aa95d4054174fff45095fd873a1 $ docker image rmi 0235ed43fc7f -f Untagged: registry.gitlab.com/security-products/secrets:6 Deleted: sha256:0235ed43fc7fb2852c76e2d6196601968ae0375c72a517bef714cd712600f894 Deleted: sha256:f05f85850cf4fac79e279d93afb6645c026de0223d07b396fce86c2f76096c1f Deleted: sha256:7432b0766b885144990edd3166fbabed081be71d28d186f4d525e52729f06b1f Deleted: sha256:2c6e3361c2ee2f43bd75fb9c7c12d981ce06df2d51a134965fa47754760efff0 Deleted: sha256:7ad7f7245b45fbe758ebd5788e0ba268a56829715527a9a4bc51708c21af1c7f Deleted: sha256:3b73a621115a59564979f41552181dce07f3baa17e27428f7fff2155042a1901 Deleted: sha256:78648c2606a7c4c76885806ed976b13e4d008940bd3d7a18b52948a6be71b60d Deleted: sha256:383d4a6dc5be9914878700809b4a3925379c80ab792dfe9e79d14b0c1d6b5fad Then I'll rerun the job to show the failure: GitLab Runner without internet access fails to pull an image from internal registry cache SCP file to GitLab instance Now, from my local machine, I will SCP the file to my GitLab instance as follows: $ gcloud compute scp secret-detection.tar INSTANCE:~ --zone=ZONE secret-detection.tar 100% 81MB 21.5MB/s 00:03 Load transferred images into offline container registry Next, I'll SSH into my VM and load the Docker image: $ gcloud compute ssh INSTANCE --zone=ZONE $ sudo docker load -i secret-detection.tar c3c8e454c212: Loading layer [==================================================>] 2.521MB/2.521MB 51e93afaeedc: Loading layer [==================================================>] 32.55MB/32.55MB e8a25e39bb30: Loading layer [==================================================>] 221.2kB/221.2kB 390704968493: Loading layer [==================================================>] 225.8kB/225.8kB 76cf57e75f63: Loading layer [==================================================>] 17.64MB/17.64MB c4c7a681fd10: Loading layer [==================================================>] 4.608kB/4.608kB f0690f406157: Loading layer [==================================================>] 24.01MB/24.01MB Loaded image: registry.gitlab.com/security-products/secrets:6 Run the scanners I'll re-run the pipeline manually and the scanner will be pulled from the cache. Once the pipeline completes, we can see the secret detection job is successful: GitLab Runner without internet access successfully pulling from internal registry cache after image loaded If you want to pull the image from a different location or you tag your images in a different way, you can edit the config as follows: include: - template: Jobs/Secret-Detection.gitlab-ci.yml variables: SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" See the offline environments documentation for more information. View scanner results Once the scanner completes on the default branch, a vulnerability report is populated with all the findings. The vulnerability report provides information about vulnerabilities from scans of the default branch. You can access the vulnerability report by navigating to the side tab and selecting Secure > Vulnerability Report: GitLab Vulnerability Report with secret detection findings The project’s vulnerability report provides: totals of vulnerabilities per severity level filters for common vulnerability attributes details of each vulnerability, presented in tabular layout a timestamp showing when it was updated, including a link to the latest pipeline We can see that two vulnerabilities were detected by the Secret Detection scanner. If we click on a vulnerability, we will be transported to its vulnerability page: GitLab Vulnerability Page showing detailed insights The vulnerability page provides details of the vulnerability, which can be used to triage and find a path to remediation. These vulnerability details include: description when it was detected current status available actions linked issues actions log filename and line number of the vulnerability (if available) severity Read more To learn more about GitLab and running security scanners in air-gapped environments, check out the following resources: GitLab Ultimate GitLab Security and Compliance Solutions GitLab Offline Deployments Documentation GitLab Application Security Documentation
Picture this: Product and Development teams are working in isolation. Product has created a 12-month roadmap and communicated it to internal stakeholders but didn't review it with their development team. Dev starts building the features planned for the upcoming sprint without considering the broader product roadmap, leading to missed opportunities to optimize timing, like running projects in parallel, accounting for team capacity, or building reusable APIs that could serve multiple initiatives. The lack of coordination results in inefficiencies and delayed value delivery. Balancing short-term wins with long-term vision isn’t easy; it requires clear communication, aligned priorities, and the right tools. In this guide, you'll learn strategies to help harmonize your Agile sprints with strategic roadmaps, tackle common challenges, and uncover actionable solutions tailored to your teams. The importance of a single source of truth A consistent single source of truth for roadmaps with longer-range goals ensures you and your teams have access to up-to-date information about the bigger picture. In practice, this means maintaining a single, regularly updated platform where all roadmap details reside rather than keeping versions of the roadmap across multiple formats, each typically with slightly different information, causing a misaligned understanding of where you're headed. Create a centralized roadmap By creating a centralized roadmap for your team, you can: communicate long-range strategy minimize miscommunication facilitate cross-functional alignment quickly adapt to changes without losing context self-serve information, reducing dependency on a single point of contact who retains the information GitLab tip: Use epics and Roadmap view to support both product planning and the transparent monitoring of delivery. The Roadmap view allows you to track progress, identify bottlenecks, and ensure alignment between high-level goals and sprint-level execution. Collaborative roadmap review practices Establish a regular review and sign-off process for roadmap updates that include Product, Engineering, and UX as part of the product trio. Collaborative reviews help you maintain alignment and minimize risk. At GitLab, I meet with my engineering manager and UX designer monthly to review and obtain sign-offs on any changes. We maintain a running sign-off on the roadmap wiki page itself that holds us accountable for keeping the schedule and provides transparency to the rest of the organization. How to extract value from review sessions To make the most of the review session, aim for the following best practices: Schedule routine reviews, monthly or quarterly, depending on how frequently the roadmap tends to fluctuate at your organization. Validate alignment between product goals, UX lead time, and technical feasibility by discussing potential risks and dependencies upfront. Validate that the roadmap reflects current organizational business objectives. Ensure that design timelines are realistic and consider research or validation needs. Confirm that the roadmap allocates time for technical preparation, such as technical spikes or investigations, and ensures alignment with broader engineering priorities. Optimize team utilization by considering capacity constraints and ensuring the sequence of work aligns with the team’s skill profile. This includes avoiding periods of underutilization or skill mismatches while effectively planning for situations like staffing level drops during holidays. Right-size scope and set appropriate expectations about what can be achieved. We all want to do it all, but perfection is the enemy of progress so prioritize what truly matters to deliver incremental value efficiently. Seek opportunities to optimize by identifying ways to iterate or increase velocity, such as adjusting the order of work to reduce dependencies or leveraging reusable components to streamline development. Encourage open dialogue about trade-offs and priorities to ensure all perspectives are considered. This collaborative approach helps identify creative solutions to challenges and builds consensus on the best path forward. GitLab tip: Use a GitLab Wiki page to complement the Roadmap feature. In the wiki, you can include expanded context about your product roadmap, such as business rationale, links to user research, RICE scores, and details about dependencies or risks. Link directly to the roadmap for easy access, and leverage the upcoming discussion threads feature to encourage async collaboration and feedback from your team. Continuous direction validation and progress measurement The goal of a product roadmap isn’t just to stay on track – it’s to deliver real value to your customers. To make space for sharing ongoing user feedback and behavioral data consider incorporating regular touchpoints across your product trio outside of sprint cycles. These sessions can be used to review insights, analyze trends, and ensure that the product roadmap continues to reflect the evolving needs of your users. By grounding roadmap updates using real user insights, you’re not only delivering on outcomes but also adapting to what really matters to your customers. The value you ship might come in the form of improved usability, reduced technical debt, or entirely new capabilities. When the product trio is aligned on the roadmap vision, they’re also aligned on the outcomes you’re working to achieve. To measure whether you’re on track to deliver those outcomes, you need to closely scope the intended results. Scope creep, like late user story additions, can delay your ability to ship value. Additionally, it’s important to identify work that was delivered but doesn’t align with the roadmap and understand why. Sprint planning Remaining aligned with your product roadmap starts with thoughtful sprint planning. Here are some best practices to keep your team on track and focused on delivering value: Clearly define, and narrowly scope, desired outcomes to ensure high confidence in delivery. Identify potential late additions or adjustments that could delay delivery, and build in buffers to maintain focus. Align on the sequence of work with your team to optimize for capacity, skill profiles, and reducing dependencies. To maintain focus and improve confidence of delivering on time, avoid planning to 100% of the team’s capacity. Leave room (10%-20%) for unknowns or new discoveries that may surface during the sprint. During the sprint Staying aligned with your roadmap during the sprint requires focus, communication, and constant evaluation. While delivering value is the goal, it’s equally important to ensure the work in progress aligns with the outcomes you’ve scoped and planned. Continuously validate the work in progress against roadmap outcomes to ensure every sprint contributes to the bigger picture. Encourage the team to regularly check if they’re still working toward the intended goals and outcomes. Maintain open communication throughout the sprint. Use daily standups or async updates to surface risks, unplanned work, or dependencies early and adjust where necessary. Be ruthless about protecting the sprint. While the urge to solve emerging problems is natural, unplanned work should be carefully evaluated to avoid derailing agreed-upon priorities. Proactively manage scope creep. If new work surfaces mid-sprint, assess whether it aligns with the current roadmap outcome’s narrowly scoped focus. While additional ideas or features may align conceptually with the broader outcome, they may not fit into the immediate plan to deliver value as soon as possible. Document these suggestions and evaluate if they should be considered as part of future iterations or as a nice-to-have for the future, rather than introducing them into the current sprint and delaying agreed-upon priorities. Sprint retros In your sprint retrospectives, take time to reflect with your team on how well you are collectively progressing toward your desired outcomes. Questions to ask: Did any unplanned work get introduced during the sprint that delayed your ability to deliver value? Identify why it happened and what adjustments can be made. Did you deliver any work that deviated from the roadmap? Discuss what led to this and what you can learn for future planning. From sprint planning through retrospectives, staying focused on delivering tangible outcomes to users and stakeholders is a team responsibility. By aligning every step of the way, you ensure that your roadmap remains a clear guide for delivering value efficiently and consistently. GitLab tip: Use burndown charts to visualize progress and detect deviations early, helping your team stay focused on delivering outcomes. Delivering roadmap outcomes with confidence Harmonizing Agile sprints with strategic roadmaps requires intentionality, team buy-in, and the proper tools. By creating a roadmap single source of truth, fostering collaborative reviews, and measuring progress towards outcomes, you can align execution with vision. With GitLab’s robust planning features, teams can turn challenges into opportunities for innovation and growth. Ready to align your sprints with your strategic roadmap? Start a free trial of GitLab today and explore the tools that can help you deliver outcomes with confidence. Learn more Agile planning content hub GitLab’s new Planner role for Agile planning teams Get to know the GitLab Wiki for effective knowledge management
This past year, over 800 community members have made more than 3,000 contributions to GitLab. These contributors include team members from global organizations like Thales, Scania, and Kitware, who are helping shape GitLab's future through the Co-Create Program — GitLab's collaborative development program where customers work directly with GitLab engineers to contribute meaningful features to the platform. Through workshops, pair programming sessions, and ongoing support, program participants get hands-on experience with GitLab's architecture and codebase while solving issues or improving existing features. "Our experience with the Co-Create Program has been incredible," explains Sébastien Lejeune, open source advocate at Thales. "It only took two months between discussing our contribution with a GitLab Contributor Success Engineer and getting it live in the GitLab release." In this post, we'll explore how customers have leveraged the Co-Create Program to turn their ideas into code, learning and contributing along the way. The Co-Create experience The GitLab Development Kit (GDK) helps contributors get started developing on GitLab. "The advice I would give new contributors is to remember that you can't break anything with the GDK," says Hook. "If you make a change and it doesn't work, you can undo it or start again. The beauty of GDK is that you can tinker, test, and learn without worrying about the environment." Each participating organization in the Co-Create Program receives support throughout their contribution journey: Technical onboarding workshop: A dedicated session to set up the GitLab Development Kit (GDK) and understand GitLab's architecture 1:1 engineering support: Access to GitLab engineers for pair programming and technical guidance Architecture deep dives: Focused sessions on specific GitLab components relevant to the issue the organization is contributing to Code review support: Detailed feedback and guidance through the merge request process Regular check-ins: Ongoing collaboration to ensure progress and address any challenges This structure ensures that teams can contribute effectively, regardless of their prior experience with GitLab's codebase or the Ruby/Go programming language. As John Parent from Kitware notes, "If you've never seen or worked with GitLab before, you're staring at a sophisticated architecture and so much code across different projects. The Co-Create Program helps distill what would take weeks of internal training into a targeted crash course." The result is a program that not only helps deliver new features but also builds lasting relationships between GitLab and its user community. "It's inspiring for our engineers to see the passion our customers bring to contributing to and building GitLab together," shares Shekhar Patnaik, principal engineer at GitLab. "Customers get to see the 'GitLab way,' and engineers get to witness their commitment to shaping the future of GitLab." Enhancing project UX with Thales When Thales identified opportunities to improve GitLab's empty project UI, they didn't just file a feature request — they built the solution themselves. Their contributions focused on streamlining the new project setup experience by simplifying SSH/HTTPS configuration with a tabbed interface and adding copy/paste functionality for the code snippets. These changes had a significant impact on developer workflows. The team's impact extended beyond the UX improvements. Quentin Michaud, PhD fellow for cloud applications on the edge at Thales, contributed to improving the GitLab Development Kit (GDK). As a package maintainer for Arch Linux, Michaud's expertise helped improve GDK's documentation and support its containerization efforts, making it easier for future contributors to get started. "My open source experience helped me troubleshoot GDK's support for Linux distros,” says Michaud. “While improving package versioning documentation, I saw that GitLab's Contributor Success team was also working to set up GDK into a container. Seeing our efforts converge was a great moment for me — it showed how open source collaboration can help build better solutions." The positive experience for the Thales team means that Lejeune now uses the Co-Create Program as "a powerful example to show our managers the return on investment from open source contributions." Advancing package support with Scania When Scania needed advanced package support in GitLab, they saw an opportunity to contribute and build it themselves. "As long-time GitLab users who actively promote open source within our organization, the Co-Create Program gave us a meaningful way to contribute directly to open source," shares Puttaraju Venugopal Hassan, solution architect at Scania. The team started with smaller changes to familiarize themselves with the codebase and review process, then progressed to larger features. "One of the most rewarding aspects of the Co-Create Program has been looking back at the full, end-to-end process and seeing how far we've come," reflects Océane Legrand, software developer at Scania. "We started with discovery and smaller changes, but we took on larger tasks over time. It's great to see that progression." Their contributions include bug fixes for the package registry and efforts to enhance the Conan package registry feature set, bringing it closer to general availability (GA) readiness while implementing Conan version 2 support. Their work and collaboration with GitLab demonstrates how the Co-Create Program can drive significant improvements to GitLab’s package registry capabilities. "From the start, our experience with the Co-Create Program was very organized. We had training sessions that guided us through everything we needed to contribute. One-on-one sessions with a GitLab engineer also gave us an in-depth look at GitLab’s package architecture, which made the contribution process much smoother," said Juan Pablo Gonzalez, software developer at Scania. The impact of the program goes beyond code — program participants are also building valuable skills as a direct result of their contributions. In the GitLab 17.8 release, both Legrand and Gonzalez were recognized as GitLab MVPs. Legrand talked about how the work she's doing in open source impacts both GitLab and Scania, including building new skills for her and her team: "Contributing through the Co-Create Program has given me new skills, like experience with Ruby and background migrations. When my team at Scania faced an issue during an upgrade, I was able to help troubleshoot because I'd already encountered it through the Co-Create Program." Optimizing authentication for high-performance computing with Kitware Kitware brought specialized expertise from their work with national laboratories to improve GitLab's authentication framework. Their contributions included adding support for the OAuth2 device authorization grant flow in GitLab, as well as implementing new database tables, controllers, views, and documentation. This contribution enhances GitLab's authentication options, making it more versatile for devices without browsers or with limited input capabilities. "The Co-Create Program is the most efficient and effective way to contribute to GitLab as an external contributor," shares John Parent, R&D engineer at Kitware. "Through developer pairing sessions, we found better implementations that we might have missed working alone." As a long-time open source contributor, Kitware particularly appreciated GitLab's approach to development. "I assumed GitLab wouldn't rely on out-of-the-box solutions at its scale, but seeing them incorporate a Ruby dependency instead of building a custom in-house solution was great,” says Parent. “Coming from the C++ world, where package managers are rare, it was refreshing to see this approach and how straightforward it could be." Building better together: Benefits of Co-Create The Co-Create Program creates value that flows both ways. "The program bridges a gap between us as GitLab engineers and our customers," explains Imre Farkas, staff backend engineer at GitLab. "As we work with them, we hear their day-to-day challenges, the parts of GitLab they rely on, and where improvements can be made. It's great to see how enthusiastic they are about getting involved in building GitLab with us." This collaborative approach also accelerates GitLab's development. As Shekhar Patnaik, principal engineer at GitLab, observes: "Through Co-Create, our customers are helping us move our roadmap forward. Their contributions allow us to deliver critical features faster, benefitting our entire user base. As the program scales, there's a real potential to accelerate development on our most impactful features by working alongside the very people who rely on them." Get started with Co-Create Ready to turn your feature requests into reality? Whether you're looking to enhance GitLab's UI like Thales, improve package support like Scania, or optimize authentication like Kitware, the Co-Create Program welcomes organizations who want to actively shape GitLab's future while building valuable open source experience. Contact your GitLab representative to learn more about participating in the Co-Create Program, or visit our Co-Create page for more information.
Imagine introducing a powerful new AI tool that boosts your team's productivity — accelerating code development, resolving issues faster, and streamlining workflows. The excitement is palpable, but questions about security and compliance quickly arise. How do you manage the risk of AI inadvertently exposing sensitive data or responding to malicious prompts? This is where prompt guardrails play a crucial role. Prompt guardrails are structured safeguards – combining instructions, filters, and context boundaries – designed to guide AI models toward secure and reliable responses. Think of them as safety rails on a bridge, working to keep data and interactions on the correct path while supporting your organization's security protocols. In this article, we'll explore how GitLab implements these guardrails, the risks they address, and their importance for security-conscious enterprises and compliance-focused teams. Why prompt guardrails matter AI models have transformed how organizations work, offering powerful tools to enhance productivity and innovation. However, this power comes with inherent risks. Without safeguards, AI systems may unintentionally disclose sensitive information, such as personally identifiable information (PII) or proprietary business data, or potentially act on malicious instructions. Prompt guardrails address these challenges by creating boundaries for AI models to access and process approved content, contributing to reduced risk of unintended data exposure or manipulation. For businesses operating under strict regulations like GDPR, prompt guardrails serve as essential protection mechanisms. More importantly, they build trust among decision-makers, end users, and customers, demonstrating GitLab's commitment to secure and responsible AI usage. With prompt guardrails in place, teams can embrace AI's potential while maintaining focus on protecting their critical assets. GitLab’s approach to prompt guardrails At GitLab, we're building AI features with security, transparency, and accountability in mind because we understand these elements are critical for our enterprise customers and their auditors. Here’s how we’re putting that into practice. Structured prompts and context boundaries Our system utilizes tags – like or – to define boundaries for AI model interactions. When users ask GitLab Duo to troubleshoot a job failure, relevant logs are encapsulated in tags. This structure guides the model to focus on specific data while working to prevent the influence from unauthorized or out-of-scope information. Filtering and scanning tools We employ tools like Gitleaks to scan inputs for secrets (API keys, passwords, etc.) before transmission to the AI. This filtering process helps minimize the potential for exposing confidential information or sending credentials into a model's prompt. Role-based insights Our guardrails support focused AI discussions while contributing to customers' compliance efforts through controlled data handling and clear documentation. Organizations can adopt AI solutions designed to align with enterprise policies and risk tolerances. Different approaches to prompt guardrails Prompt guardrails aren't one-size-fits-all solutions. Different strategies offer unique advantages, with effectiveness varying by use case and organizational requirements. GitLab combines multiple approaches to create a comprehensive system designed to balance security with usability. System-level filters: The first line of defense System-level filters serve as a proactive barrier, scanning prompts for restricted keywords, patterns, or potentially harmful content. These filters work to identify and block potential risks — such as profanity, malicious commands, or unauthorized requests — before they reach the AI model. This approach requires continuous updates to maintain effectiveness. As threats evolve, maintaining current libraries of restricted keywords and patterns becomes crucial. GitLab integrates these filters into its workflows to address potential risks at the earliest stage. Model instruction tuning: Teaching the AI to stay on track Instruction tuning involves configuring AI behavior to align with specific guidelines. Our AI models are designed to reduce potentially problematic behaviors like role play, impersonation, or generating inappropriate content. This foundation supports responses that remain informative, professional, and focused. When summarizing discussions or analyzing code, the AI maintains focus on the provided context, ideally mitigating potential deviation into unrelated topics. Sidecar or gateway solutions: Adding a layer of protection Sidecar or gateway solutions function as security checkpoints between users and AI models, processing both inputs and outputs. Like a customs officer reviewing luggage, these components help ensure only appropriate content passes through. This approach proves particularly valuable in environments requiring strict information control, such as regulated industries or compliance-driven workflows. Why GitLab combines these approaches No single strategy addresses all potential risks. GitLab's hybrid approach combines system-level filters, instruction tuning, and sidecar solutions to create a robust security framework while maintaining usability. System-level filters provide initial screening, while instruction tuning aligns AI behavior with security standards. Sidecar solutions offer additional oversight, supporting transparency and control over data flow. This combination creates a framework designed to support confident AI adoption while aiming to protect sensitive data and maintain compliance requirements. Lessons learned While prompt guardrails help to significantly reduce risks, no system is infallible. Here are some lessons we have learned along the way: Overly restrictive rules might hamper legitimate usage, frustrate developers, or slow down workflows. Striking the right balance between protecting data and providing real value is key. Threat landscapes change, as do the ways people use AI. Regular updates to guardrails support alignment with current requirements and potential threats At GitLab, we understand that no system can promise absolute security. Instead of making guarantees, we emphasize how our guardrails are designed to reduce risks and strengthen your defenses. This transparent approach builds trust by acknowledging that security is an ongoing process — one that we continuously refine to help support your organization’s evolving needs. We gather feedback from actual user scenarios to iterate on our guardrails. Real-world insights help us refine instructions, tighten filters, and improve scanning tools over time. Summary Prompt guardrails go beyond being a technical solution — they represent GitLab’s commitment to prioritizing AI security for our customers. By helping to reduce exposure, block harmful inputs, and ensure clear traceability of AI interactions, these guardrails aim to provide your teams with the confidence to innovate securely. With GitLab Duo, our structured prompts, scanning tools, and carefully tuned instructions work together to help keep AI capabilities aligned with compliance standards and best practices. Whether you’re a developer, auditor, or decision-maker, these safeguards aim to enable you to embrace AI confidently while staying true to your organization’s security and compliance goals. Learn more about GitLab Duo and get started with a free, 60-day trial today!
Conducting security scans is a regular part of any software development process. Whether scanning source code (e.g., Java, Python, or other languages), configuration files (e.g., YAML files), or container images, these scanning tools help development teams be proactive about understanding and addressing security threats. Traditionally, developers run these security scans as part of CI/CD pipelines. By including these scans in CI/CD, every change to a project will be reviewed to see if any vulnerabilities are introduced. Understanding security concerns during development helps to assure that changes are addressed before they are deployed to a live environment, but there are many additional benefits to conducting container vulnerability scans post deployment as well. GitLab's Operational Container Scanning feature allows DevSecOps practitioners to run container vulnerability scans against containers running in a Kubernetes environment. The benefits of conducting a vulnerability scan on deployed containers include regularly scanning the images for new vulnerabilities that are discovered, tracking which environments certain vulnerabilities are deployed to, and also tracking the progress of resolving these vulnerabilities. The scans can be configured to run on a regular cadence and on containers in specific namespaces on a Kubernetes cluster. The results of these scans are then sent back to GitLab projects to be viewed via the GitLab UI. To show exactly how the feature works, the next steps in this article will demonstrate how to apply the Operational Container Scanning feature using a GitLab project, sample application, and a Kubernetes cluster. Prerequisites To get started, you will need the following: GitLab Ultimate account Kubernetes cluster that meets GitLab’s Kubernetes version requirements kubectl CLI helm CLI Additionally, the walkthrough below will use a GitLab project that can be forked into a GitLab group where you have appropriate permissions to carry out the steps that follow. Deploy a sample application The first action we will carry out is to deploy a sample application to the Kubernetes cluster you will use in this tutorial. Before running the kubectl command to deploy a sample application, take a moment to make sure your KUBECONFIG is set to the cluster you would like to use. Once you are set up to use your cluster, run the following command: $ kubectl apply -f https://gitlab.com/gitlab-da/tutorials/cloud-native/go-web-server/-/raw/main/manifests/go-web-server-manifests.yaml namespace/go-web-server-dev created deployment.apps/go-web-server created service/go-web-server created Wait for all the pods to be running in the go-web-server-dev namespace by running the command below: $ kubectl get pods -n go-web-server-dev -w You should see output similar to what is shown below: NAME READY STATUS RESTARTS AGE go-web-server-f6b8767dc-57269 1/1 Running 0 18m go-web-server-f6b8767dc-fkct2 1/1 Running 0 18m go-web-server-f6b8767dc-j4qwg 1/1 Running 0 18m Once everything is running, you can set up your forked GitLab project to connect to your Kubernetes cluster and configure the Operational Container Scanning properties. Connect Kubernetes cluster In this section, you will learn how to connect a Kubernetes cluster to your GitLab project via the GitLab Agent for Kubernetes. By configuring and installing the agent on your Kubernetes cluster, you will be able to also configure Operational Container Scanning. Change the id property for GitLab’s Kubernetes agent In the forked GitLab project you are using, change the id property in the config.yaml file to match the group where you have forked the project. By doing this, you will configure the GitLab Agent for Kubernetes to pass information about your cluster back to your GitLab project. Make sure to commit and push this change back to the main branch of the forked project. Navigate to Kubernetes clusters page of the project In the GitLab UI, select the Operate > Kubernetes clusters tab of the forked project. Click the Connect a cluster (agent) button. Add the name of the agent to the input box under Option 2: Create and register an agent with the UI and then click Create and register. In this case, the name of the agent is k8s-agent since the folder under agents with the config.yaml file is named k8s-agent. Note that this folder can have any name that follows Kubernetes naming restrictions and that k8s-agent is just being used for simplicity. Install the GitLab Kubernetes agent After registering the agent, you will be asked to run a helm command shown in the GitLab UI from your command line against your Kubernetes cluster. Before running the command, make sure your KUBECONFIG is still connected to the same cluster where you deployed the sample application. After running the helm command successfully, wait for all pods to be running in the gitlab-agent-k8s-agent namespace on your cluster. You can wait for everything to be running using the following command: $ kubectl get pods -n gitlab-agent-k8s-agent -w You should see similar output to what is shown below: NAME READY STATUS RESTARTS AGE k8s-agent-gitlab-agent-v2-6bb676b6bf-v4qml 1/1 Running 0 10m k8s-agent-gitlab-agent-v2-6bb676b6bf-xt7xh 1/1 Running 0 10m Once the pods are running, your GitLab project should be connected to your Kubernetes cluster and ready to use the Operational Container Scanning feature. Before proceeding, continue running the kubectl get pods -n gitlab-agent-k8s-agent -w command to help explain concepts in the next section. Operational Container Scanning In addition to the pods for the GitLab agent running in the gitlab-agent-k8s-agent namespace, there should eventually be another pod named trivy-scan-go-web-server-dev. This pod will start and run on a regular cadence and conduct a container vulnerability scan using a tool named trivy against the go-web-server-dev namespace where the sample application deployed earlier is running. The Operational Container Scanning properties are defined in the config.yaml file used to set up the GitLab agent for Kubernetes on your cluster. The two main properties to define are cadence, which specifies how frequently to run the container vulnerability scan, and also the namespaces property nested under vulnerability_report, which defines one or more namespaces to conduct the scan on. You can see how this looks in config.yaml below: container_scanning: cadence: '*/5 * * * *' vulnerability_report: namespaces: - go-web-server-dev The cadence follows a cron format. In this case, */5 * * * * means the scan will be run every five minutes, but this can be changed to any amount of time (e.g., every 24 hours). The vulnerabilities revealed by the scan for containers running in the go-web-server-dev namespace are sent back to your GitLab project. To see the results, go to the GitLab UI and select your forked project. Select the Secure > Vulnerability report option for the project and then select the Operational vulnerabilities tab to view scan results. The scan results will include information on the severity of the common vulnerabilities and exposures (CVEs), along with the name of the image. By using the tag of the image to include the version of the deployed software along with what environment it is deployed to, you can begin to audit what known vulnerabilities exist in your Kubernetes environments and keep track of how they are being addressed by engineering teams. Watch this demo for more information: Share your feedback Adding GitLab’s Operational Container Scanning to your Kubernetes environments can help development, security, and infrastructure teams have a consistent picture of container security in Kubernetes environments across an organization. In addition to GitLab’s CI container scanning capabilities and the ability to scan containers pushed to GitLab’s container registry, GitLab has solutions at every phase of the software development lifecycle to address container security concerns. You can share your feedback on Operational Container Scanning in this forum post, which we will share with our product and engineering teams supporting this feature. You can get started with Operational Container Scanning by reading the documentation on the feature and starting a 60-day free trial of GitLab Ultimate.
This blog post is the second post in a series about GitLab Universal Automated Response and Detection (GUARD). Writing and deploying security threat detections in an organization’s security information event management platform (SIEM) is a critical component of a successful cybersecurity program. Moving from manual detection engineering to a fully automated process by implementing Detections as Code (DaC) ensures detection consistency, quality, auditing, and automated testing. At GitLab, we’ve embedded DaC capabilities into GUARD, our fully automated detection and response framework. The problem: Source control and automated tests The Signals Engineering and SIRT team at GitLab share the responsibility to create, update, and decommission threat detections in our SIEM. Maintaining a single source of truth for detections is critical to ensure detection consistency and quality standards are met. Our teams made the conscious decision to abstract the detection creation process from our SIEM, improving our issue tracking, consistency, roll-back process, and metrics. Additionally, conducting pre-commit detection tests outside of our SIEM ensured that newly created detections didn’t introduce overly false positive heavy alerts, which would require tuning or disablement while the alert was fixed. The Solution: Leverage GitLab CI/CD for detection testing and validation To address these challenges, we developed an efficient workflow using GitLab CI/CD, resulting in a streamlined and secure SIEM detection deployment process. Key components of the GUARD DaC pipeline 1. Detections stored in JSON format in a GitLab project GitLab uses the JSON format for our threat detections. The template includes essential information such as SIEM query logic, detection title, and description along with runbook page link, MITRE tactic and technique related to the detection, and other necessary details. 2. Initiating merge requests When a GitLab team member intends to create a new threat detection, update an existing one, or delete a current detection, they initiate the process by submitting a merge request (MR) in the DaC project containing the detection JSON template. Creating the MR automatically triggers a CI/CD pipeline. 3. Automated validation with CI/CD jobs Each MR contains a number of automated checks via GitLab CI/CD: Query format validation queries SIEM API to ensure detection query is valid JSON Detection fields validation validates all required fields are present, and are in the correct format New detections and detection modification trigger a number of SIEM API calls to ensure the detection does not have any errors and that no issues will be introduced into our production detection rules Detection deletion MRs trigger the pipeline to issue a SIEM API query to ensure the detection to be deleted is still active and can be deleted 4. Peer review and approval When a detection MR job completes successfully, a peer review is required to review and confirm the MR meets required quality and content standards before the detection MR can be merged. Merge request approval rules are used to trigger the peer review process. 5. Merge and final deployment After the MR is approved, it is merged into the main branch. As part of the CI/CD pipeline, an automated job executes a SIEM API command in order to perform two tasks: Create the new detection or update/delete the existing detection if needed. Extract the MITRE ATT&CK tactic and technique information related to the alert from the JSON files and transmit these details to a lookup table within the SIEM. This lookup table plays an important role in mapping our alerts to MITRE tactics and techniques, helping us improve our threat analysis and identify gaps in our detection capabilities in alignment with the MITRE framework. Note: The necessary credentials for these actions are securely stored in CI/CD variables to ensure the process remains confidential and secure. Below is a template GitLab CI/CD gitlab-ci.yml configuration file for a DaC pipeline: # --------------------------------------------------------------------------- # # GitLab CI/CD Pipeline for SIEM Detection Management # --------------------------------------------------------------------------- # image: python:3.12 # --------------------------------------------------------------------------- # # Global Configuration # --------------------------------------------------------------------------- # before_script: - apt-get update && apt-get install -y jq - pip install --upgrade pip - pip install -r requirements.txt # --------------------------------------------------------------------------- # stages: - fetch - test - process - upload # --------------------------------------------------------------------------- # # Fetch Stage # --------------------------------------------------------------------------- # fetch_changed_files: stage: fetch Script: - echo "Fetching changed files..." - git branch - git fetch origin $CI_DEFAULT_BRANCH:$CI_DEFAULT_BRANCH --depth 2000 - | if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then git diff --name-status HEAD^1...HEAD > changed-files-temp.txt else git fetch origin $CI_COMMIT_BRANCH:$CI_COMMIT_BRANCH --depth 2000 git diff --name-status ${CI_DEFAULT_BRANCH}...${CI_COMMIT_SHA} > changed-files-temp.txt fi - grep -E '\.json$' changed-files-temp.txt > changed-files.txt || true - flake8 . - pytest artifacts: paths: - changed-files.txt expose_as: 'changed_files' # --------------------------------------------------------------------------- # # Test Stage # --------------------------------------------------------------------------- # flake8: stage: test script: - echo "Running Flake8 for linting..." - flake8 . pytest: stage: test script: - echo "Running Pytest for unit tests..." - pytest artifacts: when: always reports: junit: report.xml # --------------------------------------------------------------------------- # # Process Stage # --------------------------------------------------------------------------- # process_files: stage: process script: - echo "Processing changed files..." - git clone --depth 2000 --branch $CI_DEFAULT_BRANCH $CI_REPOSITORY_URL - mkdir -p modified_rules delete_file new_file - python3 move-files.py -x changed-files.txt - python3 check-alerts-format.py artifacts: paths: - modified_rules - delete_file - new_file # --------------------------------------------------------------------------- # # Upload Stage # --------------------------------------------------------------------------- # update_rules: stage: upload script: - echo "Uploading updated rules and lookup tables..." - git fetch origin $CI_DEFAULT_BRANCH:$CI_DEFAULT_BRANCH --depth 2000 - git clone --depth 2000 --branch $CI_DEFAULT_BRANCH $CI_REPOSITORY_URL - python3 update-rules.py - python3 update-exceptions.py - python3 create_ttps_layers.py rules: - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE != "schedule" changes: - detections/**/* - exceptions/**/* The diagram below illustrates the workflow of the CI/CD process described above. graph TD; fetch[Fetch Stage: Identify Changed Files] --> test[Test Stage: Run Linting and Tests]; test --> process[Process Stage: Categorize Files]; process --> upload[Upload Stage: Update Rules and Lookup Tables]; fetch --> fetch_details[Details: Filter JSON files, Output 'changed-files.txt']; test --> test_details[Details: Run Flake8 for linting, Pytest for testing]; process --> process_details[Details: Categorize into 'modified', 'new', 'deleted', Prepare for upload]; upload --> upload_details[Details: Update repo, Update detections in SIEM and SIEM lookup table]; Benefits and outcomes Automating our detections lifecycle through a DaC CI/CD-powered workflow introduces numerous benefits to our threat detection deployment process: Automation: Automating the creation and validation of SIEM detections reduces manual errors and saves time. Enhanced security: The CI-driven workflow enforces a "least privilege" policy, ensuring consistency, peer reviews, and quality standards for creating, updating, or deleting threat detections. Efficiency: The standardized JSON detection format and automated creation expedite the deployment process. Collaboration: The MR and review process fosters collaboration and knowledge sharing among GitLab team members. Version control: Treating threat detection as code abstracts the detections from the SIEM platform they are ultimately stored in. This abstraction provides a historical record of changes, facilitates collaboration, and enables rollbacks to previous configurations if issues arise. Get started with DaC Using GitLab CI/CD and a "least privilege" policy has made our SIEM detection and alert management easier and more secure. Automation has improved efficiency and reduced risks, providing a helpful example for others wanting to improve their security and compliance. You can try this tutorial by signing up for a free 60-day trial of GitLab Ultimate.
Artificial intelligence (AI) is quickly becoming the backbone of modern software development, fueling developer efficiency and accelerating innovation. With the emergence of AI agents implementing code based on instructions from humans, we are learning that implementing AI-based features has its own unique set of security challenges. How do we protect access to the resources AI needs, protect confidentiality, and avoid privilege escalation? Few organizations are ready to answer these questions today. At GitLab, we are. We are introducing a new paradigm for identity management: composite identities. When AI agents are integrated into your DevSecOps workflows, previously simple questions become difficult to answer: Who authored this code? Who is the author of this merge request? Who created this Git commit? We found we had to start asking new questions: Who instructed an AI agent to generate this code? What context did AI need to build this feature? What were the resources AI had to access and read to generate the answer? To answer these questions, we need to understand some fundamental aspects of AI’s identity: Does an AI agent have its own distinct identity? What is the representation of this identity? How do we make it all secure? Authentication and AI identity management We are at the beginning of a paradigm shift in identity management in the software delivery lifecycle. Before the AI era, identity management was simpler. We had human user-based identities and machine-type identities using separate accounts. With the emergence of AI and agentic workflows, the distinction between these two core types of identities has blurred. AI agents are supposed to work in an autonomous way, so it makes sense to think about them as machine-type accounts. On the other hand, AI agents are usually being instructed by human users, and require access to resources the human users have access to in order to complete their tasks. This introduces significant security risks — for example, the AI may provide human users with information they should not have access to. How do we avoid privilege escalation, provide auditability, and protect confidentiality in a world with AI agents? The solution: Composite identities A composite identity is our new identity principal, representing an AI agent’s identity that is linked with the identity of a human user who requests actions from the agent. This enhances our ability to protect resources stored in GitLab. Whenever an AI agent with a composite identity attempts to access a resource, we will not only authenticate the agent itself, but also link its principal with a human user who is instructing the agent, and will try to authorize both principals before granting access to a resource. Both principals need access; otherwise, the access will be denied. If an AI agent by itself can access a project, but a human user who instructed the agent to do so cannot, GitLab will deny the access. The inverse is true as well — if a human user can access a confidential issue, but an AI agent can’t, then its service account will not be able to read the issue. We authorize access to every API request and for each resource an agent attempts to access this way. Composite identity without a request-scoped link to a human account will not be authorized to access any resource. For fully autonomous workloads we are also considering adding support for linking composite identities with other principals. Composite identity and service accounts We redesigned our authorization framework to support composite identities, allowing multiple principals to be evaluated simultaneously when determining access rights to a resource. We enhanced our security infrastructure by implementing scoped identities across our entire system — from API requests to CI jobs and backend workers. These identities are linked to an AI agent's composite identity account also through OAuth tokens and CI job tokens. This project yielded unexpected security benefits, particularly in GitLab CI, where we upgraded job tokens to signed JSON web tokens (JWTs). Additionally, we contributed code to several open source libraries to add support for scoped identities. Composite identity with GitLab Duo with Amazon Q In the GitLab 17.8 release, we made composite identity for service accounts support available for customers through our GitLab Duo with Amazon Q integration. Amazon Q Developer agent will have composite identity enforced, which will protect your confidential GitLab resources from unauthorized access. What’s next? To learn more, check out our composite identity docs.
Welcome to our "Getting started with GitLab" series, where we help newcomers get familiar with the GitLab DevSecOps platform. Knowing how to import your projects to GitLab is an essential skill to make the most of the GitLab DevSecOps platform. You’ve set up your account, invited users, and organized them based on your use case or team structure. Now, you need to bring your existing projects into GitLab and start collaborating. These projects can be local files on your computer or hosted on a different source code management platform. Let's explore the options. Importing local project files You don't want to start from scratch every time you import a project. Follow these steps to get into GitLab existing legacy projects or applications that exist without version control or use version control. Git project If Git is already initiated in your local project, create a new project in GitLab and obtain the SSH or HTTPS URL by clicking on the Code button in the top right corner of your project page. Switch to your terminal and ensure you are in your project folder: cd /project_folder Backup your existing Git origin: git remote rename origin old-origin Add the GitLab remote URL for the new origin, when using SSH: git remote add origin [git@gitlab.com](mailto:git@gitlab.com):gitlab-da/playground/abubakar/new-test-repo.git And for HTTPS: git remote add origin https://gitlab.com/gitlab-da/playground/abubakar/new-test-repo.git Then push all existing branches and tags to GitLab: git push --set-upstream origin --all git push --set-upstream origin --tags All your file project files, branches, and tags will be pushed to GitLab and you can start collaborating. Non-Git project Alternatively, if you have not initiated Git in your project, you will need to initialize Git, commit existing files, and push to GitLab as follows: git init --initial-branch=main git remote add origin git@gitlab.com:gitlab-da/playground/abubakar/new-test-repo.git git add . git commit -m "Initial commit" git push --set-upstream origin main Importing from online sources If you have your project on GitLab.com or other platforms and you want to move it to another GitLab instance (like a self-managed instance) or from another platform to GitLab.com, GitLab provides the import project feature when you want to create a new project. Importing a project migrates the project files and some other components of the project depending on the source. You can import from different sources like Bitbucket, GitHub, Gitea, and a GitLab instance, among other sources. Import sources are enabled by default on GitLab.com, but they need to be enabled for self-managed by an administrator. We will look at a few of these sources in the following sections. GitLab sources You can export projects from GitLab.com and GitLab Self-Managed instances using the Export project feature in a project’s settings. To access it: Go to your project’s settings and click into the General area. Scroll to and Expand Advanced section. Select Export project. A notification will be shown stating: “Project export started. A download link will be sent by email and made available on this page.” After the export is generated, you can follow the link contained in the email or refresh the project settings page to reveal the “Download export” option. Importing the project Click on the New project button in your target GitLab instance. Select Import project and click on GitLab Export in the list of import sources. Specify a project name and select the export file, then click Import project. An "import in progress" page will be shown and once complete, you will be redirected to the imported project. Depending on the size of your project, the import time may vary. It's important to note that not everything in a project might be exported and a few things might change after import. Review the documentation to understand the limitations. If you want to migrate a whole group instead of individual projects, the Direct Transfer method is recommended; this creates a copy of an entire group. Third-party providers GitLab supports importing from Bitbucket Cloud, Bitbucket Server, FogBugz, Gitea, and GitHub. The import process is similar across all the supported third parties — the main difference is in the method of authentication. Let's look at a few of them. GitHub There are three methods to import GitHub projects in to GitLab: Using GitHub OAuth Using a GitHub personal access token Using the API Importing using GitHub OAuth and personal access token are similar. The difference lies in how your authorize GitLab to access your repositories. The OAuth method is easier because you only need to click on the “Authorize with GitHub” button and your are redirected to your GitHub account to authorize the connection. Then the list of your projects is loaded for you to pick those you want to import. Alternatively, you will need to generate a GitHub personal access token, selecting the repo and read:org scopes, and then provide it on the "Import" page. For API imports, you can use the same personal access token with our Import REST API endpoints in your script or application. In this demo, GitLab Senior Developer Advocate Fernando Diaz explains how to import a project from GitHub using the OAuth method: You can learn about prerequisites, known issues, importing from GitHub Enterprise, and other valuable information from the GitLab import documentation. Bitbucket Importing projects from Bitbucket is similar to importing them from GitHub. While using OAuth is applicable to Bitbucket Cloud, the SaaS version of Bitbucket, you'll need to provide a URL, username, and personal access token for Bitbucket Server, the enterprise self-hosted version. Clicking on the Bitbucket Cloud option on the "Import" screen automatically takes you to Atlassian authentication for Bitbucket. You can also import Bitbucket projects using the GitLab Import API. Gitea Importing projects from Gitea requires the creation of a personal access token on the Gitea platform and providing it along with the Gitea server URL on the GitLab import page. OAuth authentication is not supported. Generic remote Git repository Where your Git provider is not supported or import is not possible using the supported methods, a repository can be imported using its accessible https:// or git:// URL. If it's not publicly accessible, you will provide the repository URL along with username and password (or access token where applicable due to multifactor authentication). This method can also be used for maintaining a copy of a remote project and keeping it in sync, i.e., mirroring. Mirroring allows you to maintain repositories across different platforms and keep them synced. This can be to separate private and public access to project while ensuring both ends have the same copy, which is useful when open-sourcing internal projects. It can also be used when working with contractors and both parties use different platforms, and access to codebase is necessary on both ends. Summary Importing and migrating between GitLab instances and from other sources is an important process that needs to be planned to ensure the expectations are clear on what gets imported and with which method. While most third-party methods import project items, including files, issues, and merge requests, some methods have known issues and limitations. The GitLab import section of the documentation has detailed information on all the supported methods that can help you plan your migration. Want to take your learning to the next level? Sign up for GitLab University courses. Or you can get going right away with a free 60-day trial of GitLab Ultimate.
Continuous deployment is a game-changing practice that enables teams to deliver value faster, with higher confidence. However, diving into advanced deployment workflows — such as GitOps, container orchestration with Kubernetes, or dynamic environments — can be intimidating for teams just starting out. At GitLab, we're committed to making delivery seamless and scalable. By enabling teams to focus on the fundamentals, we empower them to build a strong foundation that supports growth into more complex strategies over time. This guide provides essential steps to begin implementing continuous deployment with GitLab, laying the foundation for your long-term success. Start with a workflow plan Before diving into the technical implementation, take time to map out your deployment workflow. Success lies in careful planning and a methodical approach. Artifact management strategy In the context of continuous deployment, artifacts are the packaged outputs of your build process that need to be stored, versioned, and deployed. These could be: container images for your applications packages compiled binaries or executables libraries configuration files documentation packages other artifacts Each type of artifact plays a specific role in your deployment process. For example, a typical web application might generate: a container image for the backend service a ZIP archive of compiled frontend assets SQL files for database changes environment-specific configuration files Managing these artifacts effectively is crucial for successful deployments. Here's how to approach artifact management. Artifacts and releases versioning strategies A best practice to get you started with a clean structure is to establish a clear versioning strategy for your artifacts. When creating releases: Use semantic versioning (major.minor.patch) for release tags Example: myapp:1.2.3 for a stable release Major version changes (2.0.0) for breaking changes Minor version changes (1.3.0) for new features Patch version changes (1.2.4) for bug fixes Maintain a 'latest' tag for the most recent stable version Example: myapp:latest for automated deployments Include commit SHA for precise version tracking Example: myapp:1.2.3-abc123f for debugging Consider branch-based tags for development environments Example: myapp:feature-user-auth for feature testing Build artifacts retention Implement defined retention rules: Set explicit expiration timeframes for temporary artifacts Define which artifacts need permanent retention Configure cleanup policies to manage storage Registry access and authentication Secure your artifacts with proper access controls: Implement Personal Access Tokens for developer access Configure CI/CD variables for pipeline authentication Set up proper access scopes Environment strategy Consider your environments early, as they shape your entire deployment pipeline: Development, staging, and production environment configurations Environment-specific variables and secrets Access controls and protection rules Deployment tracking and monitoring approach Deployment targets Be intentional as to where and how you'll deploy, these decisions matter and the benefits and drawbacks of each should be consider: Infrastructure requirements (VMs, containers, cloud services) Network access and security configurations Authentication mechanisms (SSH keys, access tokens) Resource allocation and scaling considerations With our strategy defined and foundational decisions made, we can now translate these plans into a working pipeline. We'll build a practical example that demonstrates these concepts, starting with a simple application and progressively adding deployment capabilities. Implementing your CD pipeline A step-by-step example Let's walk through implementing a basic continuous deployment pipeline for a web application. We'll use a simple HTML application as an example, but these principles apply to any type of application. We’re also going to deploy our application as a Docker image on a simple virtual machine. This will allow us to lean on a curated image with minimum dependencies, and to ensure no environment specific requirements are unintentionally brought in. By working on a virtual machine, we won’t be leveraging GitLab’s native integrations, allowing us to work on an easier but less scalable setup to begin with. Prerequisites In this example, we’ll aim to containerize an application that we’ll run on a virtual machine hosted on a cloud provider. We’ll also test this application locally on our machine. This list of prerequisites is only needed for this scenario. Virtual machine setup Provision a VM in your preferred cloud provider (e.g., GCP, AWS, Azure) Configure network rules to allow access on ports 22, 80, and 443 Record the machine's public IP address for deployment Set up SSH authentication: Generate a public/private key pair for the machine In GitLab, go to Settings > CI/CD > Variables Create a variable called GITLAB_KEY Set Type to "File" (required for SSH authentication) Paste the private key in the Value field Define a USER variable, this is the user logging in and running the scripts on your VM Configure deployment variables Create variables for your deployment targets: STAGING_TARGET: Your staging server IP/domain PRODUCTION_TARGET: Your production server IP/domain Local development setup Install Docker on your local machine for testing deployments GitLab Container Registry access Locate your registry path: Navigate to Deploy > Container Registry Copy the registry path (e.g., registry.gitlab.com/group/project) Set up authentication: Go to Settings > Access Tokens Create a new token with registry access Token expiration: Maximum 1 year Save the token securely Configure local registry access: docker login registry.gitlab.com # The username if you are using a PAT is gitlab-ci-token # Password: your-access-token 1. Create your application Start with a basic web application. For our example, we're using a simple HTML page: body { background-color: #171321; /* GitLab dark */ } 2. Containerize your application Create a Dockerfile to package your application: FROM nginx:1.26.2 COPY index.html /usr/share/nginx/html/index.html This Dockerfile: Uses nginx as a base image for serving web content Copies your HTML file to the correct location in the nginx directory structure 3. Set up your CI/CD pipeline Create a .gitlab-ci.yml file to define your pipeline stages: variables: TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest TAG_COMMIT: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:$CI_COMMIT_SHA stages: - publish - deploy Let's break it down: TAG_LATEST is made up of three parts: $CI_REGISTRY_IMAGE is the path to your project's container registry in GitLab For example: registry.gitlab.com/your-group/your-project $CI_COMMIT_REF_NAME is the name of your branch or tag For example, if you're on main branch: /main, and if you're on a feature branch: /feature-login :latest is a fixed suffix So if you're on the main branch, TAG_LATEST becomes: registry.gitlab.com/your-group/your-project/main:latest. TAG_COMMIT is almost identical, but instead of :latest, it uses: $CI_COMMIT_SHA which is the commit identifier, for example: :abc123def456. So for that same commit on main branch, TAG_COMMIT becomes:registry.gitlab.com/your-group/your-project/main:abc123def456. The reason for having both is TAG_LATEST gives you an easy way to always get the newest version, and TAG_COMMIT gives you a specific version you can return to if needed. 4. Publish to the container registry Add the publish job to your pipeline: publish: stage: publish image: docker:latest services: - docker:dind script: - docker build -t $TAG_LATEST -t $TAG_COMMIT . - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker push $TAG_LATEST - docker push $TAG_COMMIT This job: Uses Docker-in-Docker to build images Creates two tagged versions of your image Authenticates with the GitLab registry Pushes both versions to the registry Now that our images are safely stored in the registry, we can focus on deploying them to our target environments. Let's start with local testing to validate our setup before moving to production deployments. 5. Deploy to your environment Before deploying to production, you can test locally. We just published our image to the GitLab repository, which we’ll pull locally. If you’re unsure of the exact path, navigate to Deploy > Container Registry, and you should see an icon to copy the path of your image at the end of the line for the container image you want to test. docker login registry.gitlab.com docker run -p 80:80 registry.gitlab.com/your-project-path/main:latest By doing so you should be able to access your application locally on your localhost address through your web browser. You can now add a deployment job to your pipeline: deploy: stage: deploy image: alpine:latest script: - chmod 400 $GITLAB_KEY - apk add openssh-client - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - ssh -i $GITLAB_KEY -o StrictHostKeyChecking=no $USER@$TARGET_SERVER docker pull $TAG_COMMIT && docker rm -f myapp || true && docker run -d -p 80:80 --name myapp $TAG_COMMIT This job: Sets up SSH access to your deployment target Pulls the latest image Removes any existing container Deploys the new version 6. Track deployments Enable deployment tracking by adding environment configuration: deploy: environment: name: production url: https://your-application-url.com This creates an environment object in GitLab's Operate > Environments section, providing: Deployment history Current deployment status Quick access to your application While a single environment pipeline is a good starting point, most teams need to manage multiple environments for proper testing and staging. Let's expand our pipeline to handle this more realistic scenario. 7. Set up multiple environments For a more robust pipeline, configure staging and production deployments: stages: - publish - staging - release - version - production staging: stage: staging rules: - if: $CI_COMMIT_BRANCH == "main" && $CI_COMMIT_TAG == null environment: name: staging url: https://staging.your-app.com # deployment script here production: stage: production rules: - if: $CI_COMMIT_TAG environment: name: production url: https://your-app.com # deployment script here This setup: Deploys to staging from your main branch Uses GitLab tags to trigger production deployments Provides separate tracking for each environment Here and in our next step, we’re leveraging a very useful GitLab feature: tags. By manually creating a tag in the Code > Tags section, the $CI_COMMIT_TAG gets created, which allows us to trigger jobs accordingly. 8. Create automated release notes We'll be using GitLab's release capabilities through our CI/CD pipeline. First, update your stages in .gitlab-ci.yml: stages: - publish - staging - release # New stage for releases - version - production Next, add the release job: release_job: stage: release image: registry.gitlab.com/gitlab-org/release-cli:latest rules: - if: $CI_COMMIT_TAG # Only run when a tag is created script: - echo "Creating release for $CI_COMMIT_TAG" release: # Release configuration name: 'Release $CI_COMMIT_TAG' description: 'Release created from $CI_COMMIT_TAG' tag_name: '$CI_COMMIT_TAG' # The tag to create ref: '$CI_COMMIT_TAG' # The tag to base release on You can enhance this by adding links to your container images: release: name: 'Release $CI_COMMIT_TAG' description: 'Release created from $CI_COMMIT_TAG' tag_name: '$CI_COMMIT_TAG' ref: '$CI_COMMIT_TAG' assets: links: - name: 'Container Image' url: '$CI_REGISTRY_IMAGE/main:$CI_COMMIT_TAG' link_type: 'image' For meaningful automated release notes: Use conventional commits (feat:, fix:, etc.) Include issue numbers (#123) Separate subject from body with blank line If you want custom release notes with deployment info: release_job: script: - | DEPLOY_TIME=$(date '+%Y-%m-%d %H:%M:%S') CHANGES=$(git log $(git describe --tags --abbrev=0 @^)..@ --pretty=format:"- %s") cat > release_notes.md Releases. 9. Put it all together This is what our final YAML file looks like: variables: TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest TAG_COMMIT: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:$CI_COMMIT_SHA STAGING_TARGET: $STAGING_TARGET # Set in CI/CD Variables PRODUCTION_TARGET: $PRODUCTION_TARGET # Set in CI/CD Variables stages: - publish - staging - release - version - production # Build and publish to registry publish: stage: publish image: docker:latest services: - docker:dind rules: - if: $CI_COMMIT_BRANCH == "main" && $CI_COMMIT_TAG == null script: - docker build -t $TAG_LATEST -t $TAG_COMMIT . - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker push $TAG_LATEST - docker push $TAG_COMMIT # Deploy to staging staging: stage: staging image: alpine:latest rules: - if: $CI_COMMIT_BRANCH == "main" && $CI_COMMIT_TAG == null script: - chmod 400 $GITLAB_KEY - apk add openssh-client - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - ssh -i $GITLAB_KEY -o StrictHostKeyChecking=no $USER@$STAGING_TARGET " docker pull $TAG_COMMIT && docker rm -f myapp || true && docker run -d -p 80:80 --name myapp $TAG_COMMIT" environment: name: staging url: http://$STAGING_TARGET # Create release release_job: stage: release image: registry.gitlab.com/gitlab-org/release-cli:latest rules: - if: $CI_COMMIT_TAG script: - | DEPLOY_TIME=$(date '+%Y-%m-%d %H:%M:%S') CHANGES=$(git log $(git describe --tags --abbrev=0 @^)..@ --pretty=format:"- %s") cat > release_notes.md << EOF ## Deployment Info - Deployed on: $DEPLOY_TIME - Environment: Production - Version: $CI_COMMIT_TAG ## Changes $CHANGES ## Artifacts - Container Image: \`$CI_REGISTRY_IMAGE/main:$CI_COMMIT_TAG\` EOF release: name: 'Release $CI_COMMIT_TAG' description: './release_notes.md' tag_name: '$CI_COMMIT_TAG' ref: '$CI_COMMIT_TAG' assets: links: - name: 'Container Image' url: '$CI_REGISTRY_IMAGE/main:$CI_COMMIT_TAG' link_type: 'image' # Version the image with release tag version_job: stage: version image: docker:latest services: - docker:dind rules: - if: $CI_COMMIT_TAG script: - docker pull $TAG_COMMIT - docker tag $TAG_COMMIT $CI_REGISTRY_IMAGE/main:$CI_COMMIT_TAG - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker push $CI_REGISTRY_IMAGE/main:$CI_COMMIT_TAG # Deploy to production production: stage: production image: alpine:latest rules: - if: $CI_COMMIT_TAG script: - chmod 400 $GITLAB_KEY - apk add openssh-client - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - ssh -i $GITLAB_KEY -o StrictHostKeyChecking=no $USER@$PRODUCTION_TARGET " docker pull $CI_REGISTRY_IMAGE/main:$CI_COMMIT_TAG && docker rm -f myapp || true && docker run -d -p 80:80 --name myapp $CI_REGISTRY_IMAGE/main:$CI_COMMIT_TAG" environment: name: production url: http://$PRODUCTION_TARGET This complete pipeline: Publishes images to the registry (main branch) Deploys to staging (main branch) Creates releases (on tags) Versions images with release tags Deploys to production (on tags) Key benefits: Clean reproducible, local development and testing environment Clear path to production environments with structure to build confidence in what is deployed Pattern to recover from unexpected failures, etc. Ready to scale/adopt more complex deployment strategies Best practices Throughout implementation, maintain these principles: Document everything, from variable usage to deployment procedures Use GitLab's built-in features (environments, releases, registry) Implement proper access controls and security measures Plan for failure with robust rollback procedures Keep your pipeline configurations DRY (Don't Repeat Yourself) Scale your deployment strategy What next? Here are some aspects to consider as your continuous deployment strategy matures. Advanced security measures Enhance security through: Protected environments with restricted access Required approvals for production deployments Integrated security scanning Automated vulnerability assessments Branch protection rules for deployment-related changes Progressive delivery strategies Implement advanced deployment strategies: Feature flags for controlled rollouts Canary deployments for risk mitigation Blue-green deployment strategies A/B testing capabilities Dynamic environment management Monitoring and optimization Establish robust monitoring practices: Track deployment metrics Set up performance monitoring Configure deployment alerts Establish deployment SLOs Regular pipeline optimization Why GitLab? GitLab's continuous deployment capabilities make it a standout choice for modern deployment workflows. The platform excels in streamlining the path from code to production, offering built-in container registry, environment management, and deployment tracking all within a single interface. GitLab's environment-specific variables, deployment approval gates, and rollback capabilities provide the security and control needed for production deployments, while features like review apps and feature flags enable progressive delivery approaches. As part of GitLab's complete DevSecOps platform, these CD capabilities seamlessly integrate with your entire software lifecycle. Get started today The journey to continuous deployment is an evolution, not a revolution. Start with the fundamentals, build a solid foundation, and gradually incorporate advanced features as your team's needs grow. GitLab provides the tools and flexibility to support you at every stage of this journey, from your first automated deployment to complex, multi-environment delivery pipelines. Sign up for a free, 60-day trial of GitLab Ultimate to get started with continous deployment today.
Deploying an application to the cloud often requires assistance from production or DevOps engineers. GitLab's Google Cloud integration empowers developers to handle deployments independently. In this tutorial, you'll learn how to deploy a server to Google Cloud in less than 10 minutes using Go. Whether you’re a solo developer or part of a large team, this setup allows you to deploy applications efficiently. You'll learn how to: Create a new project in GitLab Create a Go server utilizing main.go Use the Google Cloud integration to create a Service account Use the Google Cloud integration to create Cloud Run via a merge request Access your newly deployed Go server Clean up your environment Prerequisites Owner access on a Google Cloud Platform project Working knowledge of Golang Working knowledge of GitLab CI 10 minutes Step-by-step Golang server deployment to Google Cloud 1. Create a new blank project in GitLab. We decided to call our project golang-cloud-run for simplicity. 2. Create a server utilizing this main.go demo. Find the main.go demo here. // Sample run-helloworld is a minimal Cloud Run service. package main import ( "fmt" "log" "net/http" "os" ) func main() { log.Print("starting server...") http.HandleFunc("/", handler) // Determine port for HTTP service. port := os.Getenv("PORT") if port == "" { port = "8080" log.Printf("defaulting to port %s", port) } // Start HTTP server. log.Printf("listening on port %s", port) if err := http.ListenAndServe(":"+port, nil); err != nil { log.Fatal(err) } } func handler(w http.ResponseWriter, r *http.Request) { name := os.Getenv("NAME") if name == "" { name = "World" } fmt.Fprintf(w, "Hello %s!\n", name) } 3. Use the Google Cloud integration to create a Service account. Navigate to Operate > Google Cloud > Create Service account. 4. Configure the region you would like the Cloud Run instance deployed to. 5. Use the Google Cloud integration to configure Cloud Run via Merge Request. 6. This will open a merge request. Immediately merge the MR. This merge request adds a CI/CD deployment job to your pipeline definition. In our case, this is also creating a pipeline definition, as we didn’t have one before. Note: The CI/CD variables GCP_PROJECT_ID, GCP_REGION, GCP_SERVICE_ACCOUNT, GCP_SERVICE_ACCOUNT_KEY will all be automatically populated from the previous steps. 7. Voila! Check your pipeline and you will see you have successfully deployed to Google Cloud Run utilizing GitLab CI. 8. Click the Service URL to view your newly deployed server. Alternatively, you can navigate to Operate > Environments to see a list of deployments for your environments. By clicking on the environment called main, you’ll be able to view a complete list of deployments specific to that environment. Next steps To get started with developing your Go application, try adding another endpoint. For instance, in your main.go file, you can add a /bye endpoint as shown below (don’t forget to register the new handler function in main!): func main() { log.Print("starting server...") http.HandleFunc("/", handler) http.HandleFunc("/bye", byeHandler) func byeHandler(w http.ResponseWriter, r *http.Request) { name := os.Getenv("NAME") if name == "" { name = "World" } fmt.Fprintf(w, "Bye %s!\n", name) } Your main.go file should now look something like this: // Sample run-helloworld is a minimal Cloud Run service. package main import ( "fmt" "log" "net/http" "os" ) func main() { log.Print("starting server...") http.HandleFunc("/", handler) http.HandleFunc("/bye", byeHandler) // Determine port for HTTP service. port := os.Getenv("PORT") if port == "" { port = "8080" log.Printf("defaulting to port %s", port) } // Start HTTP server. log.Printf("listening on port %s", port) if err := http.ListenAndServe(":"+port, nil); err != nil { log.Fatal(err) } } func handler(w http.ResponseWriter, r *http.Request) { name := os.Getenv("NAME") if name == "" { name = "World" } fmt.Fprintf(w, "Hello %s!\n", name) } func byeHandler(w http.ResponseWriter, r *http.Request) { name := os.Getenv("NAME") if name == "" { name = "World" } fmt.Fprintf(w, "Bye %s!\n", name) } Push the changes to the repo, and watch the deploy-to-cloud-run job deploy the updates. Once it’s complete, go back to the Service URL and navigate to the /bye endpoint to see the new functionality in action. Clean up the environment To prevent incurring charges on your Google Cloud account for the resources used in this tutorial, you can either delete the specific resources or delete the entire Google Cloud project. For detailed instructions, refer to the cleanup guide. Discover more tutorials like this in our Solutions Architecture area.
AI capabilities are rapidly reshaping how teams build, secure, and deploy applications. As part of our ongoing commitment to helping you navigate the evolving marketplace, GitLab has introduced more than 440 improvements in the past three releases. We're excited to spotlight three standout features making an immediate impact on how teams approach AI-powered DevSecOps. In addition, we announced we are partnering with AWS to launch GitLab Duo with Amazon Q, combining our strengths to transform software development. We're creating an experience, together, that makes AI-powered development feel seamless and upholds the security, compliance, and reliability that enterprises require. Learn how GitLab can deliver 483% ROI over the next three years, according to Forrester Consulting. 1. Vulnerability Resolution: Streamline security remediation GitLab’s 2024 Global DevSecOps Report found that 66% of companies are releasing software twice as fast — or faster — than in previous years, as businesses strive to deliver more value to their customers than competitors. However, speed introduces risk. With security teams outnumbered by dev teams 80:1, threat actors are able to exploit applications at a record pace. Last year alone, 80% of the top data breaches stemmed from attacks at the application layer. GitLab Duo Vulnerability Resolution addresses this challenge head-on. When vulnerabilities are detected in your code, you can now access detailed information right from the vulnerability report and invoke GitLab Duo to automatically create a merge request that updates your code and mitigates the risk. While developers must review these auto-generated merge requests before merging to verify the changes, this automation significantly streamlines the remediation process. Vulnerability Resolution pairs with Vulnerability Explanation, which also recently became generally available. Vulnerability Explanation gives developers a detailed description of the vulnerability infecting their code, real-world examples of how attackers can exploit the vulnerable code, and practical suggestions for remediation. By expediting the vulnerability remediation process, your teams can focus on delivering software faster while maintaining strong security practices. With less time spent researching and remediating vulnerabilities, developers can concentrate on building features that drive business value. GitLab Duo Vulnerability Resolution is available as a GitLab Duo Enterprise add-on. 2. Model Registry: Breaking down silos between Data Science and Development teams For organizations building AI-powered applications, bridging the gap between data science and software development teams has been a persistent challenge. Data scientists and developers often work in disconnected tools and workflows, leading to friction, delays, and potential errors when deploying models to production. GitLab Model Registry directly addresses this challenge by providing a centralized hub where data science and development teams can collaborate seamlessly within their existing GitLab workflow. Built with MLflow native integration, the registry allows data scientists to continue using their preferred tools while making models and artifacts instantly accessible to the broader development team. This unified approach transforms team collaboration. Data scientists can version models, store artifacts, and document model behavior through comprehensive model cards, while developers can easily integrate these models into their applications using GitLab CI/CD pipelines for automated testing and deployment. Additionally, the Model Registry's semantic versioning and GitLab API integration enables teams to implement robust governance and automate production deployments, creating a streamlined environment where data scientists and developers can work together effectively to deliver AI-powered innovation. Model Registry is available across all tiers for SaaS and self-managed customers. See the release blog for 17.6 and documentation for more. 3. Secret Push Protection: Shift security left with proactive secret detection Teams often face a critical security challenge: Developers may hardcode sensitive information like API keys, tokens, and credentials as plain text in source code repositories, sometimes without even realizing it. This creates an easy target for threat actors and puts your organization at risk. Secret Push Protection directly addresses this problem by blocking developers from pushing code that contains secrets, significantly reducing the likelihood of a breach. It works by leveraging customizable rules to identify high-confidence secrets before they ever reach your repository. What makes this solution particularly powerful is its integration with our pipeline secret detection, creating a comprehensive defense strategy. Secret Push Protection is now generally available for all GitLab Ultimate tier and GitLab Dedicated customers. Put these features to work today At GitLab, we’re committed to making it easier for teams to build software, faster. Capabilities like GitLab Duo Vulnerability Resolution, Model Registry, and Secret Push Protection are just a few of the recent innovations we’ve delivered to help developers and security teams level up their DevSecOps workflows. To learn more, check out our releases page. Get started with these new features today with a free, 60-day trial of GitLab Ultimate.
We are excited to announce that hosted runners for GitLab Dedicated, our single-tenant SaaS solution, have transitioned from beta to limited availability, marking a significant milestone in our commitment to simplifying CI/CD infrastructure management for our customers. Streamlined CI/CD infrastructure management Managing runner infrastructure has traditionally been a complex undertaking, requiring dedicated resources and expertise to maintain optimal performance. Hosted runners for GitLab Dedicated eliminates these challenges by providing a fully managed solution that handles all aspects of runner infrastructure. This allows your teams to focus on what matters most – building and deploying great software. Key benefits Reduced operational overhead By choosing hosted runners, you can eliminate the complexity of provisioning, maintaining, and securing your runner infrastructure. Our fully managed service handles all aspects of runner operations, from deployment to updates and security patches. Automatic scaling Hosted runners automatically scale to match your CI/CD demands, ensuring consistent performance during high-traffic periods and for large-scale projects. This dynamic scaling capability means you'll always have runners available to pick up your CI/CD jobs and ensure optimal efficiency of your development teams. Cost optimization With hosted runners, you only pay for the resources you actually use. This consumption-based model eliminates the need to maintain excess capacity for peak loads, potentially reducing your infrastructure costs while ensuring resources are available when needed. Enterprise-grade security Following the same security principles as GitLab Dedicated, hosted runners provide complete isolation from other tenants and are secure by default. Jobs are executed in fully-isolated VMs with no inbound traffic allowed. This means you can maintain the highest security standards without the complexity of implementing and maintaining security measures yourself. Introducing native Arm64 support Our hosted runners now include native Arm64 support in addition to our existing x86-64 runners, offering significant advantages for modern development workflows. Enhanced performance for Arm-based development Native Arm64 runners enable you to build, test, and deploy Arm-based applications in their native environment, ensuring optimal performance and compatibility. Teams developing Docker images or services targeting Arm-based cloud platforms can see build times cut significantly, accelerating their development cycles and deployments. Cost-efficient computing Arm-based runners can significantly reduce your computing costs, due to their efficient processing architecture and lower cost per minute. For compatible jobs, this means more affordable pipeline execution. Native building capabilities With support for both x86-64 and Arm64 architectures, you can: build and test applications natively on either architecture create multi-architecture container images efficiently validate cross-platform compatibility in your CI/CD pipeline optimize your delivery pipeline for specific target platforms eliminate the performance overhead of emulation when building for Arm targets This dual-architecture support ensures you have the flexibility to choose the right environment for each specific workload while maintaining a consistent and efficient CI/CD experience across all your projects. Available runner sizes We're expanding our runner offerings to include both x86-64 and Arm64 architectures with a range of configurations. The following sizes are available: Size vCPUs Memory Storage Small 2 8 GB 30 GB Medium 4 16 GB 50 GB Large 8 32 GB 100 GB X-Large 16 64 GB 200 GB 2X-Large 32 128 GB 200 GB This expanded size support allows you to optimize your CI/CD pipeline performance based on your application's specific requirements. What's next for hosted runners We plan to release hosted runners in general availability in May 2025. The release includes compute minute visualization to help you better understand and control your CI/CD usage across your organization. We'll be expanding our hosted runners offering with several new features coming later this year: Network controls for enhanced security and compliance MacOS runners to support application development for the Apple ecosystem Windows runners for .NET and Windows-specific workloads These additions will provide even more flexibility and coverage for your CI/CD needs, allowing you to consolidate all your build and test workflows on GitLab Dedicated hosted runners. Ready to simplify your CI/CD infrastructure? Contact your GitLab representative or reach out to our sales team to learn more about hosted runners for GitLab Dedicated.
As organizations increase in size, it becomes increasingly difficult and critical to ensure that the right team members have access to the right groups and projects within their development platform. GitLab offers some powerful methods to manage user access, especially now with custom roles, but performing this at scale through a point-and-click user interface can be frustrating. However, all is not lost. You can use Security Assertion Markup Language (SAML) and System for Cross-domain Identity Management (SCIM) as a solution. (There are moments where I’m grateful for acronyms.) I was researching this topic for a particular customer, and walking through the GitLab documentation on the capabilities, but I never felt like I truly understood the integration. As is often the case, especially when dealing with integrating components, the knowledge from experience far outweighs that gained from reading or watching. In that light, I wanted to share my steps along this path and invite you all to join me. All you need is a free trial of Microsoft Azure Entra ID and GitLab Premium with a top-level group on GitLab.com. Note: This exercise produces a working integration, however, for production environments there may be necessary deviations. For example, the user account email for the identity provider (Entra ID in this case) will likely not match your GitLab account email. Creating the application in Entra ID First, go to the Entra ID admin center. Within the Applications area, select Enterprise Applications. We’re going to create a new application, and then create our own application. Figure 1: Entra ID application creation flow With our new application created, we can start configuring the single sign-on (SSO) parameters for our application. For this task, you may want to have side-by-side browser windows. One window on your Entra ID application, and another window on the SAML settings for your GitLab group. Those settings are located under Settings, then SAML SSO on the left side of your GitLab window, as shown in Figure 2. If you don’t see this option, you aren’t in the top-level group, don’t have permission to configure SAML, or don’t have GitLab Premium enabled for that group. Figure 2: GitLab SAML configuration Within your Entra ID interface, select Single sign-on and click the SAML card. Figure 3: Entra ID SAML configuration With the side-by-side view, the SAML configuration settings are on the left and the GitLab SSO settings on the right. Figure 4: Side-by-side view of Entra ID and GitLab Now we can start copying and pasting parameters. Within the Entra ID interface, select Edit within the “Basic SAML Configuration” block. The parameter sources and destination are identified in the following table. Source (GitLab) Destination (Entra ID) Identifier Identifier (Entity ID) Assertion consumer service URL Reply URL (Assertion Consumer Service URL) GitLab single sign-on URL Sign on URL (Optional) Once completed, your side-by-side view should appear similar to the following (noting the URLs are unique to your environment). Figure 5: Completed basic SAML SSO configuration Click Save within the Entra ID “Basic SML Configuration” window to save your hard work thus far. Note: You may need to click on the “X” in the upper right of the “Basic SAML Configuration” window if it doesn’t close automatically. After this window closes, you may get a popup to test single sign-on with your application. Select No, I’ll test later, because we still have more work to do (there is always more work to do). Configuring attributes and claims Within the Entra ID user interface, look for the section for “Attributes and Claims,” and click the Edit pencil icon. The first thing we want to do is modify the Unique User identifier (Name ID) value, so click on that row and set the Source attribute to user.objectid. Additionally, the Name identifier format must be updated, and set to Persistent. Figure 6: Configuring attributes and claims Save that claim configuration. Now we have additional claims to configure, but there are only three that we need here. So, feel free to go wild and delete those default four items under Additional claims, or you can edit the existing ones to match the table below. Note that these values (specifically, the Name) are case sensitive. Name Namespace Source Attribute emailaddress http://schemas.microsoft.com/ws/2008/06/identity/claims user.otheremail NameID http://schemas.microsoft.com/ws/2008/06/identity/claims user.objectid The resulting claims configuration should appear as follows. Note the use of otheremail for the “emailaddress” attribute. This was necessary for me as my primary email addresses within Entra ID are not the addresses used on GitLab.com. If you recall, when I set up my “user," I modified the contact information to include my gitlab.com email address as one of my “Other emails.” Figure 7: Configuring the claims With your attributes configured, under the Advance settings, enable Include attribute name format setting. Figure 8: Advanced claims configuration Your "Attributes and Claims" window should now look similar to Figure 9 below. Figure 9: Configured attributes and claims If you’re happy, or at least relatively content, with your configuration, click the “X” in the top right corner of the "Attributes and Claims" window to close it. Configuring and assigning users Now that we have our application configured, we need to ensure that our users have been assigned to that application. I'll assume you’re working with a test instance that does not have the same email address as what is configured within your GitLab.com namespace. So let’s go to the “Users and groups” within the Entra ID user interface for your configured application. Figure 10: Managing application users and groups Select Add user/group, and under the “Users and groups” where it says “None Selected,” click that text. Now you can select the user(s) to add to your application. These are the users that will be permitted to log into GitLab, authenticating themselves through Entra ID. Figure 11: User selection Once selected, at the bottom of that page, click Select, and at the bottom of the next, select Assign. Now you should have a user assigned to your application. Figure 12: User assigned to application Next, we need to ensure that the GitLab.com email address for that user is configured correctly. By clicking on the user itself, we can modify or configure some additional information about that user. We can see below, the User principal name, which is based on an “onmicrosoft” domain. This is not the email address I have associated with my GitLab.com account. If you recall that we set the “Email address” attribute to “otheremail,” this is where we now configure that “other” email address. Figure 13: User properties Click the option to Edit properties for the user, and click on the Contact Information heading. Here we can add other emails – more specifically, the email address utilized for your GitLab.com account. Figure 14: Configuration of alternate email address That should complete the configuration parameters that we need in Entra ID, but wait, there’s more. Within the GitLab side now, you will need to configure a couple parameters. First, you might as well enable SAML for the group as that’s kind of a key piece here. GitLab offers some additional options to disable password authentication or enforce SSO to reduce the security risks within your application, but we’ll leave those unchecked for now. Similar to the table above, we’ll need a couple things from Entra ID to configure into GitLab. Please refer to the table below. Source (Entra ID) Destination (GitLab) Login URL Identity provider single sign-on URL Thumbprint Certificate fingerprint Figure 15: GitLab SAML configuration from Entra ID Lastly, you want to configure the default membership role for users logging in via SAML. Note that the access that you set for users here will cascade down to other groups and projects within your top-level group. Therefore, I would strongly recommend NOT setting this role to be “Owner.” Either “Guest” or “Minimal Access” would be acceptable options here, depending on the security posture of your organization. For more information about what these roles can and can not do, refer to the GitLab documentation on Roles and Permissions. Now, save your work on the GitLab interface by clicking that beautiful blue Save changes button. With your GitLab settings saved, you can now test your setup. I would encourage you to do this both through the “Verify SAML Configuration” on the GitLab system as well as with the Entra ID SSO "Test" button. Troubleshooting SAML In addition to the troubleshooting steps included within GitLab documentation, I wanted to include a couple other items that I personally experienced. If you get an error stating that the SAML reference did not contain an email address, check the Claim name for your email within the “Attributes and Claims” section within your Entra ID application. With GitLab 16.7, we added support for the “2008” attribute names, and at least for the email address setting, I found the default “xmlsoap” name for the email address claim to be a disappointing failure. Another common error is “SAML Name ID and email address do not match your user account.” As you may suspect, this error is caused by a mismatch of the “NameID” and “emailaddress” attributes within the Entra ID application. This could be a misconfiguration of the “Attributes and Claims,” but it could also be that the properties of your test user don’t match your configuration. One helpful method to identify exactly what is coming through the SAML exchange is to use a SAML Tracer or SAML Message Decoder plugin with your web browser. SCIM Now that you have SAML configured to enable users to log in via your Entra ID application, let’s make sure that people are assigned to the proper group(s) upon login. This can be incredibly helpful at scale, where instead of manually identifying which groups the particular users belong to, GitLab can learn this information from your identity application, Entra ID in this case. Because SCIM utilizes groups to identify group membership, we need to create a group within Entra ID and add the relevant user(s) to the group. For this we’ll need the main administration menu for Entra ID. Figure 16: Entra ID Group configuration We’re going to create a new group and assign our user(s) to that group. So click New group and configure a new group, which only requires you to configure a “Group name.” I used the default group type of “Security.” Leave the “Membership type” as “Assigned.” From this window, we can also assign the members. Figure 17: Creating a New Entra ID Group Once you’ve added the member(s), click Create in the bottom of that window. With your group created, and the user(s) assigned to the group, we can configure SCIM. Immediately below the SAML configuration section within the GitLab UI, you’ll see the “SCIM Token” area. Here you can generate a new token, and copy the endpoint URL, both of which will be useful for the next steps. Note that if you forget or already have a SCIM token, it can be reset. Figure 18: SCIM token and endpoint within GitLab With this information saved, return to your Entra ID application configuration. Within the left side menu, you’ll find the following: Figure 19: Provisioning SCIM within Entra ID Within the "Provisioning" section, click on New Configuration, which opens a new page where that token and URL from GitLab will be used. Figure 20: New provisioning configuration Feel free to test the connection to ensure that you’ve configured the parameters properly. After testing, click on the Create button to establish the configuration and work on our mappings and settings. You may need to click the “X” in the top right corner of the panel to return to the overview configuration. Expand the “Mappings,” which includes two parameters; “Provision Microsoft Entra ID Groups” and “Provision Microsoft Entra ID Users.” SCIM group provisioning isn’t currently supported in GitLab, and although it doesn’t break the integration, keeping group provisioning enabled may cause negligible error messages. Therefore, we want to disable “Provision Microsoft Entra ID Groups,” so click that entry and set the “Enabled” field to “No.” Figure 21: Provisioning attribute mapping Save that configuration and select “Provision Microsoft Entra ID Users.” Validate that all three "Target Object Actions" are enabled, and then proceed to the “Attribute Mapping” section. Delete all existing mappings available to delete (I find this easier because attributes can’t be assigned twice), and then configure the Attribute Mappings per the following table: customappsso Attribute (Destination) Microsoft Entra ID Attribute (Source) Matching Precedence Mapping Type externalID objectId 1 Direct active Switch([IsSoftDeleted], , "False", "True", "True", "False") Expression userName mailNickname Direct name.formatted displayName Direct Emails[type eq “other”].value userPrincipalName Direct Figure 22: Editing attributes After configuring all of the attribute mappings, the result should be similar to that found in Figure 22. Figure 23: Completed attribute mapping configuration Note the use of the “other” email within the customappssso attribute. This relates back to the “other” email we configured for the user back in the Entra ID user properties. In a production situation, the emails for the SSO account and the email address for the account within GitLab should match. With your mapping complete (congratulations, Ptolemy), there are some advanced configuration settings necessary. Underneath the "Attribute Mappings," click the box for “Show advanced options.” Once this box is checked, a link called “Edit attribute list for customappsso” is revealed. Figure 24: Advanced attribute configuration Click that link, and ensure that the Name “ID” is both “Primary Key” and “Required,” and that “externalID” is also “Required.” These attributes both refer to a unique user ID generated by Entra ID. However, although the “id” itself is required, it is not consistently provided within the API calls. Therefore, GitLab relies on the “externalID” to ensure the proper connection between the Entra ID and GitLab user accounts. Figure 25: Required attribute list Save these settings, and then close the “Attribute Mapping” page with the “X” in the top right of the window. Return to the "Application Provisioning" section and click Start provisioning. Within GitLab, we need to configure the association between the group we configured within Entra ID and the level of access we want those users to have within the GitLab top-level group. Note that this association can be configured on each sub-group within GitLab for more extensive provisioning, but within GitLab, permissions flow downhill. Whatever permission you set for a user at a top-level group, or sub-group, will cascade down to all projects and groups contained therein. Within the "Settings" portion of the GitLab menu, select SAML Group Links. Here is where you’ll configure the group name and determine what access level, or role, members of the Entra ID Group will have within this particular GitLab Group. Figure 26: GitLab SAML Group link As shown in Figure 26, I’ve configured my membership to The Academy such that any users within the dev-security group from Entra ID will be granted Developer access. Note that this is a slight variation of what a typical production environment would look like. In most instances, the user account within the identity provider (Entra ID, in this case) would match the user’s corporate account email (and we wouldn’t require “other” emails). When configured properly, if the user does not already have an account on GitLab, one will be created for them tied to their SSO account. Figure 27: SAML Group Links configured Now that you’ve completed the configuration, give it a try! From another browser, preferably in private mode to ignore any cookies or other yummy artifacts, paste the link for the GitLab SSO URL found in the GitLab SAML configurations. You should be prompted to log in with your Entra ID credentials and gain the proper access to your GitLab group! Congratulations, you’ve made it! I hope you’ve learned from and appreciate the work here, and we can all rejoice in the fact that the users within the Play-Dough app can now all properly authenticate, with the right permissions, to The Academy! Don't have a GitLab account? Sign up for a free, 60-day trial today. Read more The ultimate guide to enabling SAML and SSO on GitLab.com SAML SSO for GitLab.com groups documentation
Pipeline execution policies are a newer addition to the GitLab DevSecOps platform and a powerful mechanism to enforce CI/CD jobs across applicable projects. They enable platform engineering or security teams to inject jobs into developers’ YAML pipeline definition files, guaranteeing that certain CI/CD jobs will execute no matter what a developer defines in their `.gitlab-ci.yml` file. This article will explain how to utilize pipeline execution policies to create guardrails around the stages or jobs that a developer can use in their pipeline definition. In regulated environments, this may be necessary to ensure developers adhere to a standard set of jobs or stages in their GitLab pipeline. Any job or stage that a developer adds to their pipeline that does not adhere to a corporate standard will cause the pipeline to fail. One example use case for pipeline execution policies is ensuring a security scanner job runs. Let’s say an organization has made an investment in a third-party security scanner and they have a requirement that the external scan runs before any merge is made into the main branch. Without a pipeline execution policy, a developer could easily skip this step by not including the required code in their .gitlab-ci.yml file. With a pipeline execution policy in place, a security team can guarantee the external security scanning job executes regardless of how a developer defines their pipeline. To use pipeline execution policies to enforce these restrictions requires two parts: a shell script to make calls to the GitLab API and the policy itself. This tutorial uses a bash script; if your runner uses a different scripting language, it is easy to adapt to other languages. Here is the example shell script I will use for this exercise: #!/bin/bash echo "Checking pipeline stages and jobs..." # Pull the group access token from the environment variable GROUP_ACCESS_TOKEN="$PIPELINE_TOKEN" echo "PROJECT_ID: $PROJECT_ID" echo "PIPELINE_ID: $PIPELINE_ID" if [ -z "$GROUP_ACCESS_TOKEN" ]; then echo "GROUP_ACCESS_TOKEN (MR_GENERATOR) is not set" exit 1 fi if [ -z "$PROJECT_ID" ]; then echo "PROJECT_ID is not set" exit 1 fi if [ -z "$PIPELINE_ID" ]; then echo "PIPELINE_ID is not set" exit 1 fi # Use the group access token for the API request api_url="$GITLAB_API_URL/projects/$PROJECT_ID/pipelines/$PIPELINE_ID/jobs" echo "API URL: $api_url" # Fetch pipeline jobs using the group access token jobs=$(curl --silent --header "PRIVATE-TOKEN: $GROUP_ACCESS_TOKEN" "$api_url") echo "Fetched Jobs: $jobs" if [[ "$jobs" == *"404 Project Not Found"* ]]; then echo "Failed to authenticate with GitLab API: Project not found" exit 1 fi # Extract stages and jobs pipeline_stages=$(echo "$jobs" | grep -o '"stage":"[^"]*"' | cut -d '"' -f 4 | sort | uniq | tr '\n' ',') pipeline_jobs=$(echo "$jobs" | grep -o '"name":"[^"]*"' | cut -d '"' -f 4 | sort | uniq | tr '\n' ',') echo "Pipeline Stages: $pipeline_stages" echo "Pipeline Jobs: $pipeline_jobs" # Check if pipeline stages are approved for stage in $(echo $pipeline_stages | tr ',' ' '); do echo "Checking stage: $stage" if ! [[ ",$APPROVED_STAGES," =~ ",$stage," ]]; then echo "Stage $stage is not approved." exit 1 fi done # Check if pipeline jobs are approved for job in $(echo $pipeline_jobs | tr ',' ' '); do echo "Checking job: $job" if ! [[ ",$APPROVED_JOBS," =~ ",$job," ]]; then echo "Job $job is not approve Let’s break this down a bit. The first few lines of this code perform some sanity checks, ensuring that a pipeline ID, project ID, and group access token exist. A GitLab pipeline ID is a unique numerical identifier that GitLab automatically assigns to each pipeline run. A GitLab project ID is a unique numerical identifier assigned to each project in GitLab. A GitLab group access token is a token that authenticates and authorizes access to resources at the group level in GitLab. This is in contrast to a GitLab personal access token (PAT), which is unique to each user. The bulk of the work comes from the GitLab Projects API call where the script requests the jobs for the specified pipeline. Once you have job information for the currently running pipeline, you can use a simple grep command to parse out stage and job names, and store them in variables for comparison. The last portion of the script checks to see if pipeline stages and jobs are on the approved list. Where do these parameters come from? This is where GitLab Pipeline Execution Policies come into play. They enable injection of YAML code into a pipeline. How can we leverage injected YAML to execute this shell script? Here’s a code snippet showing how to do this. ## With this config, the goal is to create a pre-check job that evaluates the pipeline and fails the job/pipeline if any checks do not pass variables: GITLAB_API_URL: "https://gitlab.com/api/v4" PROJECT_ID: $CI_PROJECT_ID PIPELINE_ID: $CI_PIPELINE_ID APPROVED_STAGES: ".pipeline-policy-pre,pre_check,build,test,deploy" APPROVED_JOBS: "pre_check,build_job,test_job,deploy_job" pre_check: stage: .pipeline-policy-pre script: - curl -H "PRIVATE-TOKEN:${REPO_ACCESS_TOKEN}" --url "https:///api/v4/projects//repository/files/check_settings.sh/raw" -o pre-check.sh - ls -l - chmod +x pre-check.sh - DEBUG_MODE=false ./pre-check.sh # Set DEBUG_MODE to true or false allow_failure: true In this YAML snippet, we set a few variables used in the shell script. Most importantly, this is where approved stages and approved jobs are defined. After the variables section, we then add a new job to the .pipeline-policy-pre stage. This is a reserved stage for pipeline execution policies and is guaranteed to execute before any stages defined in a .gitlab-ci.yml file. There is a corresponding .pipeline-policy-post stage as well, though we will not be using it in this scenario. The script portion of the job does the actual work. Here, we leverage a curl command to execute the shell script defined above. This example includes authentication if it’s located in a private repository. However, if it’s publicly accessible, you can forgo this authentication. The last line controls whether or not the pipeline will fail. In this example, the pipeline will continue. This is useful for testing – in practice, you would likely set allow_failure: false to cause the pipeline to fail. This is desired as the goal of this exercise is to not allow pipelines to continue execution if a developer adds a rogue job or stage. To utilize this YAML, save it to a .yml file in a repository of your choice. We’ll see how to connect it to a policy shortly. Now, we have our script and our YAML to inject into a developer’s pipeline. Next, let’s see how to put this together using a pipeline execution policy. Like creating other policies in GitLab, start by creating a new Pipeline Execution Policy by navigating to Secure > Policies in the left hand navigation menu. Then, choose New Policy at the top right, and select Pipeline Execution Policy from the policy creation options. For this exercise, you can leave the Policy Scope set to the default options. In the Actions section, be sure to choose Inject and select the project and file where you’ve saved your YAML code snippet. Click on Update via Merge Request at the very bottom to create an MR that you can then merge into your project. If this is your first security policy, clicking on Merge in the MR will create a Security Policy Project, which is a project to store all security policies. When implementing any type of security policy in a production environment, access to this project should be restricted so developers cannot make changes to security policies. In fact, you may also want to consider storing YAML code that’s used by pipeline execution policies in this project to restrict access as well, though this is not a requirement. .gitlab-ci.yml file. While this use case is very focused on one aspect of security and compliance in your organization, this opens the door to other use cases. For example, you may want to make group-level variables accessible to every project within a group; this is possible with pipeline execution policies. Or, you may want to create a golden pipeline and have developers add to it. The possibilities are endless. GitLab customers are finding new and exciting ways to use this new functionality every day. If you’re a GitLab Ultimate customer, try this out today and let us know how you’re using pipeline execution policies. Not a GitLab Ultimate customer? Sign up for a free 60-day trial to get started. Read more How to integrate custom security scanners into GitLab Integrate external security scanners into your DevSecOps workflow Why GitLab is deprecating compliance pipelines in favor of security policies
For businesses that handle sensitive customer information, achieving SOC 2 (System and Organization Controls 2) compliance is not just a good practice — it's often a necessity. SOC 2 is a rigorous auditing standard developed by the American Institute of Certified Public Accountants that assesses a service organization's controls related to security, availability, processing integrity, confidentiality, and privacy. While SOC 2 is not legally mandated, it has become increasingly important, in part due to breaches consistently seen in news headlines. Obtaining SOC 2 compliance allows customers to build trust with service organizations because they know their data is being properly stored and security controls have been assessed by a third party. In this guide, we'll review the requirements for obtaining SOC 2 compliance and how GitLab can help your organization meet the highest standards for application security. What requirements are set by SOC 2 The compliance process involves an audit by an independent auditor who evaluates the design and operating effectiveness of an organization's controls. This process can be very costly, and many organizations are not sufficiently prepared before an audit. With the SOC 2 audit process typically taking close to a year, it is important to establish an efficient pre-audit process. To obtain SOC 2 compliance, an organization must meet requirements based on the Trust Services Criteria: Criteria Requirements Security - Implement controls to protect against unauthorized access - Establish procedures for identifying and mitigating risks - Set up systems for detecting and addressing security incidents Availability - Ensure systems are accessible for operation as agreed - Monitor current usage and capacity - Identify and address environmental threats that could affect system availability Process integrity - Maintain accurate records of system inputs and outputs - Implement procedures to quickly identify and correct system errors - Define processing activities to ensure products and services meet specifications Confidentiality - Identify and protect confidential information - Establish policies for data retention periods - Implement secure methods for destroying confidential data after retention periods expire Privacy - Obtain consent before collecting sensitive personal information - Communicate privacy policies clearly and in plain language - Collect data only through legal means and from reliable sources Note that these requirements are not one-time achievements, but rather a continuous process. Auditors will require control effectiveness over time. How to achieve and maintain the security requirements GitLab provides several features off the board to get you started with assuring SOC 2 security needs are met: Security Requirement Addressing Feature Implement controls to protect against unauthorized access - Confidential Issues and Merge Requests - Custom Roles and Granular Permissions - Security Policies - Verified Commit - Signed Container Images - CodeOwners - Protected Branches Set up systems for detecting and addressing security incidents - Vulnerability Scanning - Merge Request Security Widget - Vulnerability Insights Compliance Center - Audit Events - Vulnerability Report Dependency List - AI: Vulnerability Explanation - AI: Vulnerability Resolution Establish procedures for identifying and mitigating risks All the above tools can be used by a security team to establish a procedure around what to do when security vulnerabilities are identified and how they are mitigated. Let’s go through each section and highlight the security features that address these requirements. Note that a GitLab Ultimate subscription and the correct Role and Permissions are required to access many of the features listed. Be sure to check out the appropriate documentation for more information. Implement controls to protect against unauthorized access Implementing robust access controls is essential for protecting an organization's assets, ensuring regulatory compliance, maintaining operational continuity, and fostering trust. GitLab allows you to implement controls to follow the principle of least privilege, securing against unauthorized access. I will briefly cover: Security policies Custom roles and granular permissions Branch protections and CodeOwners Verified commits Security policies GitLab's security policies, known as guardrails, enable security and compliance teams to implement consistent controls across their organization, helping prevent security incidents, maintain compliance standards, and reduce risk by automatically enforcing security best practices at scale. Merge request approval policy in action The following policy types are available: Scan execution policy: Enforce security scans, either as part of the pipeline or on a specified schedule Merge request approval policy: Enforce project-level settings and approval rules based on scan results Pipeline execution policy: Enforce CI/CD jobs as part of project pipelines Vulnerability management policy: Automate vulnerability management workflows Here is an example of ensuring compliance with the pipeline execution policy: Create a project that houses multiple compliance jobs. An example of a job can be to check permissions of files that are deployed. These jobs should be generic enough that they can be applied to multiple applications. Limit the project's permissions to only security/compliance officers; don’t allow developers to remove jobs. This allows for separation of duties. Inject the compliance jobs in batch to the projects where they are required. Force them to run no matter what, but allow approval from team lead to not block development. This will ensure compliance jobs are always run and cannot be removed by developers, and that your environment remains compliant. Learn how to create security policies with our security policy documentation. Custom roles and granular permissions Custom permissions in GitLab allow organizations to create fine-grained access controls beyond the standard role-based permissions, providing benefits such as: more precise access control better security compliance reduced risk of accidental access streamlined user management support for complex organizational structures Roles and permissions settings, including custom roles Learn how to create custom roles with granular permissions using our custom role documentation. Branch protections and CodeOwners GitLab helps you further control who can change your code using two key features: Branch Protection, which lets you set rules about who can update specific branches – like requiring approval before merging changes. Code Ownership, which automatically finds the right people to review code changes by matching files to their designated owners. Together, these features help keep your code secure and high-quality by making sure the right people review and approve changes. Protected branch settings Learn how to create protected branches along with CodeOwners using protected branch and codeowner documentation. Verified commits When you sign your commits digitally, you prove they really came from you, not someone pretending to be you. Think of a digital signature like a unique stamp that only you can create. When you upload your public GPG key to GitLab, it can check this stamp. If the stamp matches, GitLab marks your commit as Verified. You can then set up rules to reject commits that aren't signed, or block all commits from users who haven't verified their identity. Commit signed with verified signature Commits can be signed with: SSH key GPG key Personal x.509 certificate Learn more about verified commits with our signed commits documentation. Set up systems for detecting and addressing security incidents Setting up systems for detecting and addressing security incidents is vital for maintaining a robust security posture, ensuring regulatory compliance, minimizing potential damages, and enabling organizations to respond effectively to the ever-evolving threat landscape. GitLab provides security scanning and vulnerability management for the complete application lifecycle. I will briefly cover: Security scanning and vulnerability management Software bill of materials System auditing and security posture review Compliance and security posture oversight Security scanning and vulnerability management GitLab provides a variety of different security scanners that cover the complete lifecycle of your application: Static Application Security Testing (SAST) Dynamic Application Security Testing (DAST) Container Scanning Dependency Scanning Infrastructure as Code (IaC) Scanning Coverage-guided Fuzzing Web API Fuzzing These scanners can be added to your pipeline via the use of templates. For example, to run SAST and dependency scanning jobs in the test stage, simply add the following to your .gitlab-ci.yml: stages: - test include: - template: Jobs/Dependency-Scanning.gitlab-ci.yml - template: Jobs/SAST.gitlab-ci.yml These jobs are fully configurable via environment variables and using GitLab job syntax. Once a pipeline kicks off, the security scanners run and detect vulnerabilities in the diff between the current branch and the target branch. The vulnerability can be seen in a merge request (MR), providing detailed oversight before the code is merged to the target branch. The MR will provide the following information on a vulnerability: description status severity evidence identifiers URL (if applicable) request/response (if applicable) reproduction assets (if applicable) training (if applicable) code flow (if using advanced SAST) MR view of introduced vulnerability Developers can use this data to remediate vulnerabilities without slowing down security team workflows. Developers can dismiss a vulnerability with reasoning, speeding up the review process, or they can create a confidential issue to track the vulnerability. If the code in an MR is merged to the default (usually production-level) branch, then the vulnerability report is populated with the security scanner results. These results can be used by security teams to manage and triage the vulnerabilities found in production. Vulnerability report with Batch Status setting When clicking on a vulnerability description within the vulnerability report, you are provided with the vulnerability page, which contains the same vulnerability data as the MR, allowing for a single source of truth when assessing impact and performing remediation. From the vulnerability page, GitLab Duo AI features can be used to explain the vulnerability and also create an MR to remediate, speeding up resolution time. Learn more about the security scanners included with GitLab and how to manage vulnerabilities in our application security documentation. Software bill of materials GitLab can create a detailed list of everything your software uses – kind of like an ingredients list for your code. This list, called a software bill of materials (SBOM), shows you all the external code your project depends on, including the parts you directly use and their own dependencies. For each item, you can see which version you're using, what license it has, and whether it has any known security problems. This helps you keep track of what's in your software and spot potential risks. Group-level dependency list (SBOM) Learn how to access and use the dependency list with our dependency list documentation. System auditing and security posture review GitLab keeps track of everything that happens in your system such as who made changes, what they changed, and when they did it. Think of it like a security camera for your code. This record helps you: spot any suspicious activity show regulators you're following the rules figure out what happened if something goes wrong see how people are using GitLab All of this information is stored in one place, making it easy to review and investigate when needed. For example, you can use audit events to track: who changed the permission level of a particular user for a GitLab project, and when who added a new user or removed a user, and when Project-level audit events Learn more about audit events, see the audit events documentation. Compliance and security posture oversight GitLab's Security Dashboard works like a control room that shows you all your security risks in one place. Instead of checking different security tools separately, you can see all their findings together on one screen. This makes it easy to spot and fix security problems across all your projects. Group-level security dashboard Learn more about security dashboards with our security dashboard documentation. Establish procedures for identifying and mitigating risks Vulnerabilities go through a specific lifecycle. For example, a part of the procedure can be to require approval for any vulnerable code to be merged to protected branches using security policies. Then the procedure can state that vulnerable code detected in production must be prioritized, assessed, remediated, and then validated: The criteria for prioritization can be by the severity of the vulnerability provided by GitLab scanners. The assessment can be done using exploitation details provided by the AI: Vulnerability Explanation. Once the vulnerability is remediated, then it can be validated using built-in GitLab regression tests and scanners. While every organization's needs are different, leveraging GitLab as a platform, risks can be quickly identified and addressed with reduced risk when compared to using a sprawl of disparate tools. Best practices for SOC 2 compliance Establish a strong security culture: Foster a culture of security awareness and accountability throughout your organization. Document everything: Maintain thorough documentation of policies, procedures, and controls. Automate where possible: Use automation tools to streamline compliance processes and reduce errors. Communicate effectively: Keep stakeholders informed about your compliance efforts. Seek expert guidance: Consider partnering with a qualified consultant to assist with your SOC 2 journey. Achieving SOC 2 compliance is a significant undertaking, but the benefits are undeniable. By demonstrating your commitment to application security and operational excellence, you can build trust with customers, enhance your reputation, and gain a competitive edge in the marketplace. Read more To learn more about GitLab and how we can help achieve SOCv2 compliance while enhancing your security posture, check out the following resources: GitLab Ultimate GitLab Security and Compliance Solutions GitLab Application Security Documentation GitLab DevSecOps Tutorial Project
Supply chain security is a critical concern in software development. Organizations need to verify the authenticity and integrity of their software packages. This guide will show you how to implement a secure CI/CD pipeline for Python packages using GitLab CI, incorporating package signing and attestation using Sigstore's Cosign. You'll learn: Why sign and attest your Python packages? Pipeline overview Complete pipeline implementation: Setting up the environment Environment configuration Configuration breakdown The 6 stages Building Signing Verification Publishing Publishing signatures Consumer verification Why sign and attest your Python packages? Here are four reasons to sign and attest your Python packages: Supply chain security: Package signing ensures that the code hasn't been tampered with between build and deployment, protecting against supply chain attacks. Compliance requirements: Many organizations, especially in regulated industries, require cryptographic signatures and provenance information for all deployed software. Traceability: Attestations provide a verifiable record of build conditions, including who built the package and under what circumstances. Trust verification: Consumers of your package can cryptographically verify its authenticity before installation. Pipeline overview Ensuring your code's integrity and authenticity is necessary. Imagine a pipeline that doesn't just compile your code but creates a cryptographically verifiable narrative of how, when, and by whom your package was created. Each stage acts as a guardian, checking and documenting the package's provenance. Here are six stages of a GitLab pipeline that ensure your package is secure and trustworthy: Build: Creates a clean, standard package that can be easily shared and installed. Signing: Adds a digital signature that proves the package hasn't been tampered with since it was created. Verification: Double-checks that the signature is valid and the package meets all our security requirements. Publishing: Uploads the verified package to GitLab's package registry, making it available for others to use. Publishing Signatures: Makes signatures available for verification. Consumer Verification: Simulates how end users can verify package authenticity. Complete pipeline implementation: Setting up the environment Before we build our package, we need to set up a consistent and secure build environment. This configuration ensures every package is created with the same tools, settings, and security checks. Environment configuration Our pipeline requires specific tools and settings to work correctly. Primary configurations: Python 3.10 for consistent builds Cosign 2.2.3 for package signing GitLab package registry integration Hardcoded package version for reproducibility Note about versioning: We've chosen to use a hardcoded version ("1.0.0") in this example rather than deriving it from git tags or commits. This approach ensures complete reproducibility and makes the pipeline behavior more predictable. In a production environment, you might want to use semantic versioning based on git tags or another versioning strategy that fits your release process. Tool requirements: Basic utilities: curl, wget Cosign for cryptographic signing Python packaging tools: build, twine, setuptools, wheel Configuration breakdown variables: PYTHON_VERSION: '3.10' PACKAGE_NAME: ${CI_PROJECT_NAME} PACKAGE_VERSION: "1.0.0" FULCIO_URL: 'https://fulcio.sigstore.dev' REKOR_URL: 'https://rekor.sigstore.dev' CERTIFICATE_IDENTITY: 'https://gitlab.com/${CI_PROJECT_PATH}//.gitlab-ci.yml@refs/heads/${CI_DEFAULT_BRANCH}' CERTIFICATE_OIDC_ISSUER: 'https://gitlab.com' PIP_CACHE_DIR: "$CI_PROJECT_DIR/.pip-cache" COSIGN_YES: "true" GENERIC_PACKAGE_BASE_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${PACKAGE_NAME}/${PACKAGE_VERSION}" We use caching to speed up subsequent builds: cache: paths: - ${PIP_CACHE_DIR} Building: Crafting the package Every software journey begins with creation. In our pipeline, the build stage is where raw code transforms into a distributable package, ready to travel across different Python environments. The build process creates two standardized formats: a wheel package (.whl) for quick, efficient installation a source distribution (.tar.gz) that carries the complete code Here's the build stage implementation: build: extends: .python-job stage: build script: - git init - git config --global init.defaultBranch main - git config --global user.email "ci@example.com" - git config --global user.name "CI" - git add . - git commit -m "Initial commit" - export NORMALIZED_NAME=$(echo "${CI_PROJECT_NAME}" | tr '-' '_') - sed -i "s/name = \".*\"/name = \"${NORMALIZED_NAME}\"/" pyproject.toml - sed -i "s|\"Homepage\" = \".*\"|\"Homepage\" = \"https://gitlab.com/${CI_PROJECT_PATH}\"|" pyproject.toml - python -m build artifacts: paths: - dist/ - pyproject.toml Let's break down what this build stage does: Initializes a Git repository (git init) and configures it with basic settings Normalizes the package name by converting hyphens to underscores, which is required for Python packaging Updates the package metadata in pyproject.toml to match our project settings Builds both wheel and source distribution packages using python -m build Preserves the built packages and configuration as artifacts for subsequent stages Signing: The digital notarization If attestation is the package's biography, signing is its cryptographic seal of authenticity. This is where we transform our package from a mere collection of files into a verified, tamper-evident artifact. The signing stage uses Cosign to apply a digital signature as an unbreakable seal. This isn't just a stamp — it's a complex cryptographic handshake that proves the package's integrity and origin. sign: extends: .python+cosign-job stage: sign id_tokens: SIGSTORE_ID_TOKEN: aud: sigstore script: - | for file in dist/*.whl dist/*.tar.gz; do if [ -f "$file" ]; then filename=$(basename "$file") cosign sign-blob --yes \ --fulcio-url=${FULCIO_URL} \ --rekor-url=${REKOR_URL} \ --oidc-issuer $CI_SERVER_URL \ --identity-token $SIGSTORE_ID_TOKEN \ --output-signature "dist/${filename}.sig" \ --output-certificate "dist/${filename}.crt" \ "$file" fi done artifacts: paths: - dist/ This signing stage performs several crucial operations: Obtains an OIDC token from GitLab for authentication with Sigstore services Processes each built package (both wheel and source distribution) Uses Cosign to create a cryptographic signature (.sig) for each package Generates a certificate (.crt) that proves the signature's authenticity Stores both signatures and certificates alongside the packages as artifacts Verification: The security checkpoint Verification is our final quality control gate. It's not just a check — it's a security interrogation where every aspect of the package is scrutinized. verify: extends: .python+cosign-job stage: verify script: - | failed=0 for file in dist/*.whl dist/*.tar.gz; do if [ -f "$file" ]; then filename=$(basename "$file") if ! cosign verify-blob \ --signature "dist/${filename}.sig" \ --certificate "dist/${filename}.crt" \ --certificate-identity "${CERTIFICATE_IDENTITY}" \ --certificate-oidc-issuer "${CERTIFICATE_OIDC_ISSUER}" \ "$file"; then failed=1 fi fi done if [ $failed -eq 1 ]; then exit 1 fi The verification stage implements several security checks: Examines each package file in the dist directory Uses Cosign to verify the signature matches the package content Confirms the certificate's identity matches our expected GitLab pipeline identity Validates our trusted OIDC provider issued the certificate Fails the entire pipeline if any verification check fails, ensuring only verified packages proceed Publishing: The controlled release Publishing is where we make our verified packages available through GitLab's package registry. It's a carefully choreographed release that ensures only verified, authenticated packages reach their destination. publish: extends: .python-job stage: publish script: - | cat ~/.pypirc [distutils] index-servers = gitlab [gitlab] repository = ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi username = gitlab-ci-token password = ${CI_JOB_TOKEN} EOF TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token \ twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi \ dist/*.whl dist/*.tar.gz The publishing stage handles several important tasks: Creates a .pypirc configuration file with GitLab package registry credentials Uses the GitLab CI job token for secure authentication Uploads both wheel and source distribution packages to the GitLab PyPI registry Makes the packages available for installation via pip Publishing signatures: Making verification possible After publishing the packages, we must make their signatures and certificates available for verification. We store these in GitLab's generic package registry, making them easily accessible to users who want to verify package authenticity. publish_signatures: extends: .python+cosign-job stage: publish_signatures script: - | for file in dist/*.whl dist/*.tar.gz; do if [ -f "$file" ]; then filename=$(basename "$file") curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" \ --fail \ --upload-file "dist/${filename}.sig" \ "${GENERIC_PACKAGE_BASE_URL}/${filename}.sig" curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" \ --fail \ --upload-file "dist/${filename}.crt" \ "${GENERIC_PACKAGE_BASE_URL}/${filename}.crt" fi done The signature publishing stage performs these key operations: Processes each built package to find its corresponding signature files Uses the GitLab API to upload the signature (.sig) file to the generic package registry Uploads the corresponding certificate (.crt) file Makes these verification artifacts available for downstream package consumers Uses the same version and package name to maintain the connection between packages and signatures Consumer verification: Testing the user experience The final stage simulates how end users will verify your package's authenticity. This stage acts as a final check and a practical example of the verification process. consumer_verification: extends: .python+cosign-job stage: consumer_verification script: - | git init git config --global init.defaultBranch main mkdir -p pkg signatures pip download --index-url "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi/simple" \ "${NORMALIZED_NAME}==${PACKAGE_VERSION}" --no-deps -d ./pkg pip download --no-binary :all: \ --index-url "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi/simple" \ "${NORMALIZED_NAME}==${PACKAGE_VERSION}" --no-deps -d ./pkg failed=0 for file in pkg/*.whl pkg/*.tar.gz; do if [ -f "$file" ]; then filename=$(basename "$file") sig_url="${GENERIC_PACKAGE_BASE_URL}/${filename}.sig" cert_url="${GENERIC_PACKAGE_BASE_URL}/${filename}.crt" curl --fail --silent --show-error \ --header "JOB-TOKEN: ${CI_JOB_TOKEN}" \ --output "signatures/${filename}.sig" \ "$sig_url" curl --fail --silent --show-error \ --header "JOB-TOKEN: ${CI_JOB_TOKEN}" \ --output "signatures/${filename}.crt" \ "$cert_url" if ! cosign verify-blob \ --signature "signatures/${filename}.sig" \ --certificate "signatures/${filename}.crt" \ --certificate-identity "${CERTIFICATE_IDENTITY}" \ --certificate-oidc-issuer "${CERTIFICATE_OIDC_ISSUER}" \ "$file"; then failed=1 fi fi done if [ $failed -eq 1 ]; then exit 1 fi This consumer verification stage simulates the end-user experience by: Creating a clean environment to test package installation Downloading the published packages from the GitLab PyPI registry Retrieving the corresponding signatures and certificates from the generic package registry Performing the same verification steps that end users would perform Ensuring the entire process works from a consumer's perspective Failing the pipeline if any verification step fails, providing an early warning of any issues Summary This comprehensive pipeline provides a secure and reliable way to build, sign, and publish Python packages to GitLab's package registry. By following these practices and implementing the suggested security measures, you can ensure your packages are appropriately verified and safely distributed to your users. The pipeline combines modern security practices with efficient automation to create a robust software supply chain. Using Sigstore's Cosign for signing and attestation, along with GitLab's built-in security features, you can provide users with trustworthy cryptographically verified packages. Get started on your security journey today with a free 60-day trial of GitLab Ultimate. Learn more Documentation: Use Sigstore for keyless signing and verification Streamline security with keyless signing and verification in GitLab Annotate container images with build provenance using Cosign in GitLab CI/CD
DevSecOps streamlines software development by allowing teams to ship features quickly and providing short feedback cycles for customers. These short feedback cycles can be used to monitor the impact of a feature from the time it is shipped and to inform developers and product managers about the success or failure of a given deployment. GitLab, as an agnostic DevSecOps platform, can act as an integration point for different CI/CD tools that often contribute to user-facing functionality. For example, the vulnerability report, which displays all detected vulnerabilities, is visible as a single functionality, but the data in the report may come from a number of different tools in various pipelines. In a heterogeneous Static Application Security Testing (SAST) setup we find two potential sources of vulnerability deduplication: Code volatility refers to the reintroduction of vulnerabilities in a constantly changing code base. Double reporting refers to duplication introduced by multiple tools that are reporting the same vulnerability. GitLab addresses these two sources of duplication by means of the Advanced Vulnerability Tracking feature, which identifies and deduplicates vulnerabilities in a constantly changing code base. Advanced Vulnerability Tracking leverages contextual information provided by generated syntax-trees to scope vulnerabilities and generates location fingerprints for vulnerabilities that are less fragile across code changes in comparison to other tracking methods. In a recent study, we demonstrated that our vulnerability tracking approach was 30% more effective than traditional, line-based vulnerability tracking where are used to fingerprint vulnerabilities. This means that advanced vulnerability tracking reduces the manual effort of auditing vulnerabilities by 30%. In addition, our study suggested that the positive effect of our vulnerability tracking method increases over time. The preprint of our study "A scalable, effective and simple Vulnerability Tracking approach for heterogeneous SAST setups based on Scope+Offset" will be presented at the 47th International Conference on Software Engineering (Software Engineering in Practice Track) 2025. Lucas Charles, Jason Leasure, and Hua Yan contributed to this article and study.
You can subscribe to this RSS to get more information