Introduction

A Pipeline Script is a programmatic definition of a software delivery pipeline, typically written in a domain-specific language (DSL) or supported scripting language like Groovy, YAML, or Bash. These scripts are the backbone of CI/CD automation, allowing teams to define how software should be built, tested, packaged, and deployed in a repeatable and version-controlled way.

Pipeline scripts are often used in tools like:

  • Jenkins (Declarative/Scripted Groovy)
  • GitHub Actions (YAML)
  • GitLab CI (YAML)
  • CircleCI (YAML)
  • Azure Pipelines (YAML)
  • Bitbucket Pipelines (YAML)

Why Use a Pipeline Script?

Traditional CI tools required point-and-click configuration via dashboards. This often led to:

  • Configuration drift
  • Inconsistent environments
  • Lack of visibility and version control

With pipeline scripts, the pipeline becomes:

  • Code: Stored in source control (Git)
  • Auditable: Change history available
  • Shareable: Reused across teams
  • Portable: Easily recreated in other environments

Common Pipeline Script Features

FeatureDescription
StagesLogical sections (build, test, deploy)
StepsCommands or scripts executed in each stage
ConditionsOnly run stages under certain criteria
Matrix BuildsRun across different OSes, versions
Environment VariablesSet and use secrets, paths, settings
ArtifactsStore and transfer build outputs
ParallelismRun jobs concurrently
TriggersDefine when pipeline runs (push, PR, cron)

Example 1: Jenkins Declarative Pipeline (Groovy)

pipeline {
  agent any

  environment {
    NODE_ENV = 'production'
  }

  stages {
    stage('Checkout') {
      steps {
        checkout scm
      }
    }

    stage('Install') {
      steps {
        sh 'npm install'
      }
    }

    stage('Test') {
      steps {
        sh 'npm test'
      }
    }

    stage('Build') {
      steps {
        sh 'npm run build'
      }
    }

    stage('Deploy') {
      when {
        branch 'main'
      }
      steps {
        sh './deploy.sh'
      }
    }
  }
}

This Groovy-based script defines:

  • 5 stages
  • A branch-specific deployment
  • Environmental variable

Example 2: GitHub Actions Workflow (YAML)

name: Node CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3
      - name: Set up Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'
      - run: npm install
      - run: npm test
      - run: npm run build

Highlights:

  • Triggered on push or PR
  • Installs dependencies
  • Runs tests and builds app

Example 3: GitLab CI/CD (YAML)

stages:
  - build
  - test
  - deploy

build:
  stage: build
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - dist/

test:
  stage: test
  script:
    - npm test

deploy:
  stage: deploy
  only:
    - main
  script:
    - ./scripts/deploy.sh

GitLab’s pipeline script defines 3 stages and a conditional deployment.

Scripted vs Declarative Pipelines (Jenkins)

TypeCharacteristics
DeclarativeStructured, easy to read, validates syntax
ScriptedGroovy-based logic, flexible, more powerful

Declarative example:

pipeline {
  agent any
  stages {
    stage('Build') {
      steps { sh 'make build' }
    }
  }
}

Scripted example:

node {
  stage('Build') {
    sh 'make build'
  }
}

Reusability in Pipeline Scripts

TechniqueTool
IncludesGitLab (include:), CircleCI (orbs:)
Shared LibrariesJenkins (vars/, src/)
Reusable WorkflowsGitHub Actions (reusable workflows)
Helm/Kustomize TemplatesKubernetes deployment logic
Scripts DirectoryBash scripts under /scripts/

Environment Variables and Secrets

Example in GitHub Actions:

env:
  NODE_ENV: production

jobs:
  build:
    steps:
      - run: echo $NODE_ENV

Secrets:

env:
  API_KEY: ${{ secrets.API_KEY }}

In Jenkins:

environment {
  API_KEY = credentials('my-api-key')
}

Artifact Handling

Artifacts are files you want to store or pass between stages.

  • Jenkins: archiveArtifacts artifacts: 'dist/**'
  • GitHub: actions/upload-artifact
  • GitLab:
artifacts:
  paths:
    - dist/

Used for build outputs, test reports, binaries, etc.

Pipeline Script Triggers

Trigger TypeExamples
Push/PRTrigger on code changes
Schedulecron jobs (nightly builds)
Tag ReleaseRun on version tagging
Manual/ApprovalRequire human interaction
API/WebhookTrigger from external system

Debugging Pipelines

ProblemSolution
Failing script commandUse verbose logging (set -x)
Missing dependenciesEnsure pre-steps install all packages
Secret not injectedCheck vault/secret manager permissions
Stage skippedRecheck conditional logic (if, when)
Workflow not triggeredVerify on: or trigger: rules

Best Practices

PracticeBenefit
Store pipelines in code repositoryVersioned and traceable
Break into logical stagesEasier debugging
Use caching wiselySpeed up builds
Avoid code duplicationUse includes, templates, shared libs
Parameterize environmentsUse input vars and secrets
Handle failures gracefullyRetry logic, try/catch, notifications
Secure sensitive infoVaults, secrets manager, masked variables
Use linting and validatorsCatch YAML/Groovy errors early

Advanced Features

  • Dynamic Matrix Jobs
    Build/test across multiple OSes or Node versions
  • Conditional Execution
    Only run tests on modified files
  • Slack/Email Notifications
    Alert on failed builds or deployments
  • Parallel Execution
    Run tests or builds simultaneously to save time
  • Resource Locking
    Prevent race conditions when deploying shared infrastructure

Summary

TopicExplanation
PurposeDefine CI/CD pipelines as code
LanguageYAML, Groovy, Bash, etc.
Key ToolsJenkins, GitHub Actions, GitLab CI, CircleCI
BenefitsVersion control, automation, scalability
Common ElementsStages, steps, environment vars, triggers
Best PracticesDRY, secure secrets, error handling, caching

Related Keywords

  • CI/CD Pipeline
  • Declarative Pipeline
  • Groovy DSL
  • Jenkinsfile
  • Pipeline as Code
  • Shared Library
  • Step Function
  • Workflow Automation
  • YAML Workflow
  • YAML Syntax