Dependency management for SFDX projects

I have long searched for a solution to install SFDX plugins per project, and I might have finally found one. If you are wondering who I am, I’m a freelance DevOps Consultant, Advisor, and independent DevOps Engineer for the Salesforce platform.

My role requires me to work on multiple Salesforce DX projects simultaneously.

The projects are unique, and so are the scripts I have developed to set up orgs for ensuring quality assurance and performing deployments. At a minimum, this might be a Bash script glueing together some standard SFDX commands.

On the other hand, other projects are more complex using third-party SFDX plugins provided by the community or even tailored, private SFDX plugins.

That’s why we should at least document the compatibility in a README like this:

This project requires SFDX version 7.171.0 and SFDX Data Move Utility (SFDMU) version 4.26.8.

Introducing my SFDX installation plugin methods

Disclaimer: The names “John Doe” and the company ACME are fictitious examples.

Here’s my global SFDX installation, which I use for managing orgs:

$ sfdx --version
sfdx-cli/7.192.2 darwin-arm64 node-v18.12.1

Here is a local SFDX installation I used in one of my SFDX projects:

$ cd $HOME/Projects/acme-inc
$ npx sfdx --version
sfdx-cli/7.171.0 darwin-arm64 node-v18.12.1

$ sfdx plugins
sfdx-plugin-auth-url 1.0.5
sfdx-plugin-source-read 1.1.4

I ‘pinned’ the version of SFDX for the project because the scripts I wrote are known to be compatible with this SFDX version. It’s no secret that I’m a fan of SFDX plugins and use them often.

Here are the plugins I used in this specific ‘ACME’ project:

$ SFDX_DATA_DIR="$(pwd)" sfdx plugins
sfdx-browserforce-plugin 2.10.0 (2.10.0)
sfdx-plugin-source-read 1.1.3 (1.1.3)
sfdmu 4.26.8 (4.26.8)

You might have already spotted the ‘tricks’ to apply when using the local installation – “npx” and “SFDX_DATA_DIR="$(pwd)"”

To use project-specific SFDX and SFDX plugins, you can follow these steps:

• Prefix SFDX commands with “npx”
• Set the environment variable “SFDX_DATA_DIR” to the path of the current directory

It is important to note the pwd is a command to get the current directory. Be aware that you also need more information in your “package.json.”

export SFDX_DATA_DIR="$(pwd)"
sfdx plugins install sfdmu
sfdx sfdmu --help

Why “local dependencies”

In the Node.js ecosystem, defining all your dependencies in your project's “package.json” file is common practice.

$ tree -L 1 .
.
├── README.md
├── package.json
└── yarn.lock

Let’s have a look at an example of a “package.json” declaring that developers need a CLI tool named “cowsay”:

{
  "name": "acme-inc",
  "version": "0.0.0-development",
  "devDependencies": {
    "cowsay": "1.5.0"
  },
  "scripts": {
    "moo": "cowsay moooooooo"
  }
}

Developers will then be able to set up a development environment on their machine by using a single command. Any disadvantages? Yes. Your “node_modules” directory in every project will be extensive, which wastes disk space.

yarn install

You can now access the locally installed “cowsay” CLI in two ways:

# through scripts defined in package.json
yarn run moo
# or using npx
npx cowsay miau

It might look like a simple tool and “cowsay” isn’t actively developed. So, the benefits of ‘pinning’ this version might not be as crucial as with other tools.

But looking at SFDX with its weekly releases, I’d argue that scripts developed two months ago, for example, might need some adjustments to work with the latest SFDX version.

The SFDX team does a great job, but SFDX is a colossal project where things sometimes go wrong. That is why breaking changes is necessary to make progress.

Let’s now have a look at how this works for SFDX.

How it works

First, we need to initialize a Node Package Manager (NPM) or Yarn project in your existing SFDX project:

yarn init
echo "node_modules/" >> .gitignore

Installing SFDX

Install “sfdx-cli” as a dependency once. This will ‘pin’ the version in “package.json” and “yarn.lock.”

yarn add --dev --exact sfdx-cli

yarn install
npx sfdx --version

From now on, everyone can set up a development environment and have the exact version of SFDX I have just defined.

Installing SFDX plugins

Here is the revelation I recently discovered – oclif plugins are located in a ‘data’ directory. For SFDX, this can be configured using the “SFDX_DATA_DIR” environment variable.

So, in theory, all you need to do is set the environment variable – “SFDX_DATA_DIR="$(pwd)"” – when running the install command.

SFDX_DATA_DIR="$(pwd)" sfdx plugins install sfdx-plugin-source-read

For Windows PowerShell users, you will execute the following command:

$env:SFDX_DATA_DIR=.
sfdx plugins install sfdx-plugin-source-read

However, this command currently breaks the formatting of your “package.json,” so you can do the following two steps manually instead:

yarn add --dev --exact sfdx-plugin-source-read

You will then add the following to your “package.json” command:

"oclif": {
    "schema": 1,
    "plugins": [
      {
        "name": "sfdx-plugin-source-read",
        "tag": "1.1.3",
        "type": "user"
      }
    ]
  }

Using this approach

If you want to use this approach, remember the following points:

• When calling SFDX commands, use “npx sfdx“ instead of “sfdx”
• When calling SFDX plugin commands, use “SFDX_DATA_DIR="$(pwd)" npx sfdx mysfdxplugin.”

Setting the environment variable can also be achieved using direnv – a tool that reads “.envrc” and “.env” files when using a terminal.

You can then document these steps in your “README.md.”

Dependency management background information to keep in mind

NPM And NPX

Another innovative feature is built into NPX. If you haven’t installed the command, it’ll ask you to download and execute it temporarily. This makes it a handy tool for running one-off commands.

The Oclif “Data Dir”

By default, SFDX plugins are installed somewhere in your “HOME” directory rather than your project directory.

You can find out the value of the configuration using “sfdx --dev-debug.”

$ sfdx --dev-debug 2>&1 | grep "\\sdata:"
2023-03-20T15:15:04.142Z sfdx                      data: /Users/john.doe/.local/share/sfdx

This is the default behavior of the oclif “data dir” configuration. On MacOS, it appears as “~/.local/share/sfdx.”

The command SFDX plugin install, “myplugin,” will use Yarn to install the SFDX plugin. This is why the directory looks like a standard Node.js project.

$ tree ~/.local/share/sfdx
/Users/john.doe/.local/share/sfdx
├── node_modules
├── package.json
└── yarn.lock

2 directories, 2 files

Previously I also installed my own SFDX plugins locally – you can find out more about this on StackExchange. However, this approach does not work for arbitrary SFDX plugins.

Time to install your scripts

My simple guide is optional for developers and teams to utilize. But in my experience, keeping it simple is the better choice. After all, as the saying goes: “Simplicity is the glory of expression.”