dotenv considered harmful

The practice of using env files is so prevalent in the industry and few developers ever stop to doubt it, but in fact, accessing environment variables from different places across your application is akin to accessing a database directly from different places across your application - it's a clear violation of software engineering best practices, and I frequently find myself explaining to my clients why using .env files or the dotenv npm package is a bad idea. In this post, I will try to break down the anti patterns I find in common practices for managing configuration, and suggest better solutions.

First and foremost, let's discuss why we need to pass configuration using environment variables. And actually, I think that this is a good idea. Environment variables are platform-independent, they support injecting values from outside the system, and it's become the standard in modern deployment infrastructure, such as Kubernetes or Docker. It's the local development environment where developers often find themselves having to create .env files, share them with each other and generally create an informal way of bootstrapping development. This becomes clear when a new developer joins the team, and she needs to go around foraging in Slack and email for the latest version of a working .env file.

Why do we need .env files at all? do we really have so many config values that they need to be packed in files and passed around? I think that no single application should rely on more than a small set of configuration variables. And usually these variables are closely related to each other, such as "how do I connect to the database" (username, password, url) or "how do I access a remote service" (token, url) - so, in general, topology and credentials. You might ask "what about all other config? connection pool size? boolean flags controlling how the app behaves?", to which I'll answer - these are not configuration. They are code. They should be committed to source control. What's worse, if you have an .env file, people will tend to add more stuff to it, which probably needs to be code and not config, because that is the path of least resistance. Just put in the .env file, then access it anywhere using process.env, right?

Wrong. Because you're now coupling what values I need from the config to how I get these values, which violates the SRP, makes it harder for you to keep track of what config you actually need, and makes the system more cumbersome to test. What to do instead? my recommendation is to read all config values explicitly from the environment on system startup, put them in a Config object, and pass that object around. If you use a modern language that supports static duck typing, you can define a Config type for each configurable part of the system, and have the big Config object conform to that type:

// main.ts
type Config = {
  dbUrl: string;
  dbPassword: string;
  dbUser: string;
  barServiceUrl: string;
  barServiceToken: string;
  httpPort: number;
}

const config: Config = {
  dbUrl: process.env.DB_URL,
  dbPassword: process.env.DB_PASSWORD,
  dbUser: process.env.DB_USER,
  barServiceUrl: process.env.BAR_SERVICE_URL,
  barServiceToken: process.env.BAR_SERVICE_TOKEN,
  httpPort: process.env.HTTP_PORT,
}

const fooRepo = new MongoFooRepo(config);
const barAdapter = new HttpBarAdapter(config);
const app = new App(fooRepo, barAdapter);
app.start(config.httpPort);

// foo.repo.ts
type Config = {
  dbUrl: string;
  dbPassword: string;
  dbUser: string;
}

class MongoFooRepo {
  constructor(private config: Config) {}
  
  ...
}

// bar.adapter.ts
type Config = {
  barServiceUrl: string;
  barServiceToken: string;
}

class HttpBarAdapter {
  constructor(private config: Config) {}
  
  ...
}

As your app matures, you might find that stuff like credentials can't be stored in environment variables anymore (for instance, because environment variables appear as cleartext in the process list) and you'll have to change the logic that populates the Config object. It will still read the URLs from the environment, but credentials now have to come from some sort of secret manager or vault. And while you could technically read them from said secure storage and populate process.env, this is convoluted and redundant.

So in production, config is populated to environment variables by the deployment system. What about local development? we surely need an .env file here, right? Wrong again! If you start your local environment using something like Docker Compose, then Docker is also starting your database, and may also start fake versions of your collaborating microservices (Bar Service in my example above). In which case, all topology and credentials can be hard-coded in the local development docker-compose.yml file (which can and should also be used in the e2e test suite):

version: '3.9'  
services:   
  mongo:     
    image: mongo
    ports:       
      - 27017:27017     
    environment:       
      - MONGO_INITDB_ROOT_USERNAME=user       
      - MONGO_INITDB_ROOT_PASSWORD=pass
      
  fake-bar: 
    build: ./fake-bar
    environment:
      - HTTP_PORT: 3001
      
  app:
    build: ./app
    environment:
      DB_URL: mongodb://mongo:27017/some-database-name
      DB_USER: user
      DB_PASSWORD: pass
      BAR_SERVICE_URL: http://fake-bar:3001
      BAR_SERVICE_TOKEN: some-value # fake has no auth
      HTTP_PORT: 3000

One issue to note is that we're duplicating properties in the global Config type and the private Config types used by different components. In the example above, for instance, the MongoFooRepo Config type explicitly duplicate the dbUrl, dbUser and dbPassword properties. I prefer it this way, because the cost of duplication is repaid by decoupling the entry point from the Config types. If you prefer a DRYer version, you could do the following:

// main.ts
import { Config as MongoConfig } from './foo.repo.ts';
import { Config as BarHttpConfig } from './bar.adapter.ts';

type Config = MongoConfig & BarHttpConfig

...

// foo.repo.ts
export type Config = {
  dbUrl: string;
  dbPassword: string;
  dbUser: string;
}

...

// bar.adapter.ts
export type Config = {
  barServiceUrl: string;
  barServiceToken: string;
}

...

In summary: environment variables are a good and common solution to passing topology and credentials from the runtime environment to your app. But they must remain an implementation detail, and accessing them should be limited to a single place in the system - usually the entry point. Make an effort to only pass credentials and topology as config, while other values might better be simple constants in your codebase. For local development, use Docker Compose with a .yml file that wholly contains all environment variables; for production, provide the values in the deployment infrastructure.

Finally:

npm uninstall dotenv

Good luck!