Dependency scan with Droid Exec

A weekly GitHub Actions workflow that sweeps for committed secrets, audits dependencies for known vulnerabilities, proposes safe upgrades, tests them without credentials, and opens a pull request.

This recipe runs once a week in GitHub Actions. Droid sweeps the repository for committed secrets and proposes upgrades for dependencies with known vulnerabilities. A separate job installs and tests those upgrades, and a third job opens one pull request with the results.

How it works

The workflow is split into three jobs so that no credential is present while dependency or test code runs:

  • scan holds FACTORY_API_KEY and read-only repository access. Droid writes a secret report without --auto, so it can only read files. It then edits package.json at --auto low, which allows file edits but blocks package installs and test runs.
  • verify holds no credentials. It applies Droid's changes, updates the lockfile, installs dependencies, and runs the tests.
  • open-pr holds PR_TOKEN. It applies the verified changes, then commits, pushes, and opens the pull request. It runs no repository or dependency code.

Every checkout that doesn't push uses persist-credentials: false, so the job's GitHub token isn't left in the Git configuration for later steps to read.

The example uses npm. For other ecosystems, swap the audit, install, and test commands, such as pip-audit, cargo audit, or govulncheck.

Prerequisites

  • Add FACTORY_API_KEY to the repository's Actions secrets. Generate a key from the Factory settings page.
  • Add a PR_TOKEN secret that holds a fine-grained personal access token or a GitHub App token with Contents and Pull requests write access to the repository. GitHub doesn't start other workflows for pull requests opened with the default GITHUB_TOKEN, so with a PR_TOKEN your normal CI runs on the pull request. Leave the token's Workflows permission off, so a tampered patch can't change your workflow files.
Note

Use a dedicated scanner, such as gitleaks or GitHub secret scanning, as your primary defense. Those tools match known credential formats exhaustively and don't send secret values anywhere. The Droid sweep sends the files it reads, including any real secrets, to the model, and it can miss some. Treat it as a second pass for credentials that pattern-based scanners don't recognize, or remove the step if your policy doesn't allow it.

The workflow

.github/workflows/droid-dependency-scan.yml
name: Droid dependency and secret scan
 
on:
  schedule:
    - cron: '0 9 * * 1' # Mondays at 09:00 UTC
  workflow_dispatch:
 
permissions:
  contents: read
 
jobs:
  scan:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    outputs:
      changed: ${{ steps.patch.outputs.changed }}
    steps:
      - uses: actions/checkout@v7
        with:
          persist-credentials: false
 
      - name: Install Droid CLI
        run: |
          curl -fsSL https://app.factory.ai/cli | sh
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
 
      - name: Secret sweep (read-only)
        continue-on-error: true
        env:
          FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
        run: |
          droid exec "Scan the tracked files in this repository for committed
          secrets: API keys, tokens, private keys, passwords, and connection
          strings. Ignore obvious placeholders and test fixtures.
          For each finding, report the file, line, and type of secret.
          Never print the secret value itself.
          Respond with a Markdown list, or 'No secrets found.'" \
            > secret-report.md
 
      - name: Upload secret report
        uses: actions/upload-artifact@v7
        with:
          name: secret-report
          path: secret-report.md
          retention-days: 14
 
      # npm audit reads the lockfile and runs no package scripts.
      - name: Audit dependencies
        id: audit
        run: |
          npm audit --json > audit.json || true
          if ! jq -e '.metadata.vulnerabilities' audit.json > /dev/null; then
            reason=$(jq -r '.error.summary // "no audit results"' \
              audit.json 2>/dev/null || true)
            echo "::error::npm audit failed: $reason"
            exit 1
          fi
          total=$(jq '.metadata.vulnerabilities.total // 0' audit.json)
          echo "total=$total" >> "$GITHUB_OUTPUT"
 
      - name: Propose upgrades
        if: steps.audit.outputs.total != '0'
        env:
          FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
        run: |
          droid exec --auto low "Read audit.json, the output of npm audit.
          For each vulnerable direct dependency, change its version in
          package.json to the lowest version that fixes the vulnerability.
          For a vulnerable transitive dependency, add an overrides entry.
          Avoid major version upgrades unless they are the only fix; list
          those in the report instead of applying them. Do not edit the
          lockfile.
          Only read and edit files. Do not run shell commands. A later job
          updates the lockfile and runs the tests, so don't list those as
          follow-ups. Write a summary of the vulnerabilities, the changes you
          made, and what still needs a human to dependency-report.md." \
            || echo "::warning::droid exec exited $?. Keeping its edits."
 
      - name: Save changes
        id: patch
        run: |
          git add package.json
          git diff --cached --binary > changes.patch
          if [ -s changes.patch ]; then
            echo "changed=true" >> "$GITHUB_OUTPUT"
          fi
 
      - name: Upload dependency report
        if: always()
        uses: actions/upload-artifact@v7
        with:
          name: dependency-report
          path: |
            dependency-report.md
            audit.json
          if-no-files-found: warn
          retention-days: 14
 
      - name: Flag unresolved vulnerabilities
        if: >-
          steps.audit.outputs.total != '0' &&
          steps.patch.outputs.changed != 'true'
        env:
          TOTAL: ${{ steps.audit.outputs.total }}
        run: |
          if [ -f dependency-report.md ]; then
            cat dependency-report.md >> "$GITHUB_STEP_SUMMARY"
          fi
          echo "::error::npm audit found $TOTAL vulnerabilities Droid" \
            "couldn't fix. See the dependency-report artifact."
          exit 1
 
      - uses: actions/upload-artifact@v7
        if: steps.patch.outputs.changed == 'true'
        with:
          name: droid-changes
          path: |
            changes.patch
            dependency-report.md
 
  verify:
    needs: scan
    if: needs.scan.outputs.changed == 'true'
    runs-on: ubuntu-latest
    outputs:
      tests: ${{ steps.test.outputs.result }}
    steps:
      - uses: actions/checkout@v7
        with:
          persist-credentials: false
 
      - uses: actions/download-artifact@v8
        with:
          name: droid-changes
 
      # Add your runtime setup here, such as actions/setup-node.
      - name: Update lockfile without running package scripts
        run: |
          git apply changes.patch
          npm install --package-lock-only --ignore-scripts
          git add package.json package-lock.json
          git diff --cached --binary > verified.patch
 
      - uses: actions/upload-artifact@v7
        with:
          name: droid-verified
          path: |
            verified.patch
            dependency-report.md
 
      - name: Install and test without credentials
        id: test
        run: |
          if npm ci && npm test; then result=passed; else result=failed; fi
          echo "result=$result" >> "$GITHUB_OUTPUT"
 
  open-pr:
    needs: verify
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          token: ${{ secrets.PR_TOKEN }}
 
      - uses: actions/download-artifact@v8
        with:
          name: droid-verified
 
      - name: Open pull request
        env:
          GH_TOKEN: ${{ secrets.PR_TOKEN }}
          TESTS: ${{ needs.verify.outputs.tests }}
        run: |
          touch dependency-report.md
          git apply verified.patch
          branch="droid/dependency-updates-$(date +%Y%m%d)-$GITHUB_RUN_ID"
          git config user.name "github-actions[bot]"
          git config user.email \
            "41898282+github-actions[bot]@users.noreply.github.com"
          git checkout -b "$branch"
          git add package.json package-lock.json
          git commit -m "chore(deps): weekly security updates"
          git push -u origin "$branch"
          draft=false
          if [ "$TESTS" != "passed" ]; then
            draft=true
            cat >> dependency-report.md <<'EOF'
 
          Install or tests failed for these upgrades in the verify job.
          Check the workflow run before you mark this ready.
          EOF
          fi
          gh pr create --draft="$draft" \
            --title "Weekly dependency security updates" \
            --body-file dependency-report.md

The dependency report and raw audit output are always uploaded as the dependency-report artifact. If the audit finds vulnerabilities but Droid can't fix any of them, for example because every fix needs a major version upgrade, the scan job fails, the report is added to the run summary, and no pull request is opened. That way, a vulnerability that needs a person still turns the weekly run red. If npm audit can't run at all, for example because there's no lockfile, the scan job fails at the audit step instead of reporting a clean result.

At --auto low, droid exec exits with an error if Droid tries a command that needs more autonomy, even after it has finished editing. The Propose upgrades step keeps the edits and logs a warning instead of failing, so the verify job decides whether the changes are good. The Secret sweep step uses continue-on-error for the same reason: a failed sweep shows as a step error but doesn't stop that week's upgrades.

When the install or the tests fail, the workflow still opens the pull request, as a draft, so the proposed upgrades aren't lost. If the lockfile update itself fails, for example with an ERESOLVE peer dependency conflict or a version that doesn't exist, the verify job fails and no pull request opens. Droid's package.json changes are still in the droid-changes artifact. The verify job builds and uploads the patch right after the lockfile update, before npm ci runs any package code, so install scripts and tests can't change what gets pushed. Only package.json and package-lock.json are staged, so workflow files and other paths never reach the pull request. Still review the lockfile diff as carefully as package.json.

Example pull request

For a repository that pinned lodash@4.17.20 and minimist@1.2.5, the pull request changed only package.json and package-lock.json, and its body included a table like this one:

PackageBeforeAfterWhy this version
minimist1.2.51.2.6Lowest version outside the vulnerable range.
lodash4.17.204.18.14.18.0 is the lowest fix, but npm marks it deprecated.

Where the secret report goes

The secret report goes to a workflow artifact, not the pull request body or an issue, so finding locations aren't posted where more people can see them. Artifacts are only private in a private repository: in a public repository, anyone signed in to GitHub can download them. For a public repository, remove the secret sweep and its upload step and rely on GitHub secret scanning, whose alerts only people with admin or security access to the repository can see. Download the report from the workflow run, rotate any real credential it lists, and then remove the secret from the code. Rewriting git history doesn't make a leaked credential safe again, so rotate first.

To block new secrets before they reach the repository, turn on GitHub push protection.

Adapt the workflow

  • Fail on findings. Add this job, so the run turns red when the report starts with anything other than "No secrets found.". It checks only the first line, because Droid may add notes about placeholders it skipped. Keep it a separate job: a failing step inside scan would also skip verify and open-pr, so that week's upgrade pull request would never open. The !cancelled() condition runs the check even when scan fails on unresolved vulnerabilities, so a vulnerability error can't hide a secret finding.

    YAML
    secret-check:
      needs: scan
      if: ${{ !cancelled() }}
      runs-on: ubuntu-latest
      steps:
        - uses: actions/download-artifact@v8
          with:
            name: secret-report
        - name: Fail on secret findings
          run: head -n 1 secret-report.md | grep -qx 'No secrets found.'
  • Monorepos. Run the audit and proposal steps once per workspace with --cwd, or use a job matrix. See Batch and parallel patterns.

  • No PR token. If you can't add PR_TOKEN, use github.token in the open-pr job, grant it contents: write and pull-requests: write, and allow Actions to create pull requests under Settings → Actions → General. Your CI won't run on those pull requests until someone closes and reopens them or pushes a commit.

For a deeper review of the code itself, including injection, auth, and supply-chain risks, schedule a Security Review audit alongside this workflow.