Config Management Roundup

Quite the problem—Drupal config management on local and prod.

Drupal 8 can be punctilious: it will simply refuse to work if there's a mismatch between database and config objects on the filesystem.

So... how do we manage two sets of configurations—one for local, and one for production?

The Problem

I have modules like varnish installed on production that shouldn't be enabled on local, and modules that are enabled on local that shouldn't be enabled on production.

I have uninstalled varnish_purger on local. When I run drush cex on local, varnish_purger is marked for uninstall in the config and it's config files are removed. I can't deploy this config to prod, or varnish_purger will be uninstalled. But I have to export field definitions and other settings for deploy.

Settings.php

In settings.php, you can override config variables like so:

$config['system.performance']['fast_404']['exclude_paths'] = '/\/(?:styles)\/|(?:system\/files)\/|favicon\.ico|apple-touch-icon(?:-precomposed)?\.png|manifest\.json|IEconfig\.xml|browserconfig\.xml/';

You can override default/settings.php by placing the following at the bottom of settings.php and overriding settings in a settings.local.php file that should be gitignored.

if (file_exists(__DIR__ . '/settings.local.php')) {
  include __DIR__ . '/settings.local.php';
}

However, modules cannot be disabled by this method.

drushrc.php

You can make a small module uninstall script in drushrc.php:

$options['shell-aliases']['local-pmu'] = '!drush pmu varnish_purger varnish_purge_tags memcache purge purge_drush purge_tokens purge_ui purge_queuer_coretags -y';

drush local-pmu will now uninstall all the modules listed above. However, when a config export is done, those module uninstalls and removed configs will get exported to drush.

Plain Drush

I heard tell that you could add modules whose config would not be exported to drush. But this option has been removed:

$ drush cex --skip-modules=varnish_purger
Unknown option: --skip-modules.  See `drush help config-export` for available options. To suppress this error, add the option --strict=0.                                     [error]
$ drush --version
 Drush Version: 8.1.11  

Likewise the following syntax in drushrc.php didn't work for me:
$command_specific['config-export']['skip-modules'] = array('devel');

It seems these functions are being deprecated in favor of contrib-space solutions.

CMI tools

CMI tools is a drush plugin that allows you to add a list of ignored config files, and use drush cexy and drush cimy to import/export config instead. When combined with drush --skip-modules, this seemed perfect, but I couldn't get ignored modules and config files to work, so I skipped this drush extension. Additionally, this functionality is replicated by Config Ignore below.

Config Ignore

This module lets you ignore certain configuration files. Config Ignore is perfect for allowing site editors to, say, modify system.site configuration--which contains that site's name, slogan, and email—on your live site without creating config changes that must be exported manually.

This is great, but you cannot use it to have specific modules enabled or disabled on prod and local, because module installed/uninstalled status is governed by a file that cannot be ignored according to module docs: core.extensions.yml.

Config Partial Export

Config Partial Export allows you to export a tarball of recently changed files. I could see using this module. Basically, you'd set up sync directories for prod and local.
Config export with different sync directories in settings.php:

$config_directories = array(
  'prod' => '../config/prod',
  'local' => '../config/local',
);

Then you'll see this option:

$ drush cex
Choose a destination.
 [0]  :  Cancel
 [1]  :  prod   
 [2]  :  local

There are now two sets of configurations. You'd then manually manage changes with the Config Partial Export module and then dump those configs directly into the folder in question. There is something I like about the control you have in this workflow, for simpler sites.

This is untested, but let's say I enable the Coffee module on local and want that to go to production. I would first make sure the only change I had intended was on deck. I would drush cpex and then copy the contents of the tarball to config/production. Then I would modify config/prod/core.extension.yml to enable the module. Then I would I would drush cex local -y. Then, after deploying code to prod, I could drush cim prod on prod.

Config split: Installation

I ended up using Config Split. It's a more robust and automated config management tool, and looks like it could even make it into core, judging by comments I see on d.o. The tutorial I pulled the most from was by Jeff Geerling, including the comments. I suggest reading it. I'm using a slightly more complex configuration. You can read about a simpler configuration here.

Enable config_split and config_filter:

composer require drupal/config_split drupal/config_filter && drush en config_split config_filter

This module was somewhat difficult for me to get my head around at first, because of the definition of "blacklist" given, "Configuration listed here will be removed from the sync directory and saved in the split directory instead." I read that "graylisting" is somewhat unpredictable, so I won’t cover it here. I would guess that graylisting is for a module that was always installed on all environments, but whose configuration might differ from environment to environment. The canonical version is stored in the main config, and overrides are stored in different environments.

So let's use a concrete example to illuminate blacklisting. I'm creating three environments: dev, prod, and local.

If I'm on local, I want coffee enabled. But it should be disabled on prod. On prod and dev, I want varnish_purger enabled, but I want that disabled on local.

Config Split: Exporting

First step. As recommended by geerlinguy, start on prod (or the environment that has all your extra modules enabled), enable config_split and config_filter, and create your three environments, dev, prod, and local. Then export them on prod in the same step that you export the enabling of the config_split module. Create directories in your sync directory dev, prod, and local and point to those folders in each of the settings. Leave other settings untouched.

*Note:* Make sure to clear your cache after every config change:

drush cr

Export the settings:

drush cex

Commit, and then pull the database down to local and dev.

If you don't, you could run into the following strange chicken/egg problem where even though config_split was already enabled you get:
Configuration config_split.config_split.prod depends on the Configuration split module that will not be installed

But broader necessity of modifying some settings on prod is that you can't manage settings for modules that are not installed on your machine unless their settings have already been blacklisted, so, for example, I have to configure varnish_purger on prod and coffee on local, then commit the changes.

For instance, if I enable stage_file_proxy and do not yet run drush cex then the config object, stage_file_proxy.settings, will not yet appear in the UI for management. One way to work around this is by pasting the names of config objects into the text field. Searching through the multiple-select list for configs can be tedious.

The reason for leaving other settings intact is that config files cannot be removed from the settings until config_split is enabled. The important thing is to deploy config_split, config_filter, and blank settings with all modules enabled in one step, and in the next step deploy actual settings.

Now., here's how you'd configure local:

After I do that, run:

drush cr
drush csex local # Note the 's'

This will copy the blacklisted files to the local directory.

Now run:

drush cex # If you're running drush < 8.1.11 you may need to run `drush csex` here.

This will implement the blacklist and delete the coffee configuration file from the main config directory.

You can skip drush csex local and just run drush cex, but you may want to take your time and understand what's happening.

Now, blacklist varnish_purger module settings on both dev and prod and export.

Config Split: Importing

When you import a configuration, you're updating the drupal database configuration from the config file system. When you import using config_split and specify an environment, you're pulling in the global configuration as well as the config files in the specified directory. Unless, I am given to understand, a file has been graylisted and an override has been placed in the environment you're currently on. Then your environment's config should override the global config.

To use config_split, you need to do one of two things:
1. Run drush cim normally and add the following to your settings.php for each environment, this example pertaining to local:

$config['config_split.config_split.dev']['status'] = TRUE;
$config['config_split.config_split.local']['status'] = TRUE;
$config['config_split.config_split.prod']['status'] = FALSE;

When this is done properly, you'll see:

Note: I like to disable the splits explicitly in the main settings.php file, then enable them in settings.local.php files. Make sure, in this case, that that the default disabling happens before the inclusion of the settings.local.php files so that the code in settings.local.php overrides the values set previously in the main settings.php file.

2. Or, you can run drush csim {environment} instead of drush cim.

Either works.

Now if you wish to import a configuration, and you've got your config_split.config set, you just run drush cim and the global config plus your blacklisted settings will be imported.

Config Split: Module Management

EDIT: As a result of this comment below, I have updated the following.

Now, I had been scratching my head for a little while, trying to figure out how to manage the enabling and disabling of modules. I did some tests to see if importing or exporting configurations was somehow using the config_split.config_spit.{env} to override the core.extensions file and enable/disable modules.

Here's what happens: config_split can enable modules, but it cannot disable modules. I was originally confused by this and recommended adding core.extensions to each of your splits and blacklisting it.

This is not necessary.

Enter Amazing-Town

When you have your environments set in settings.php, you can enable and disable modules with impunity. When you export, config files will go to environments you've specified. If there are multiple versions of a config file, say one in local and one on dev, and you are on local, you will only be modifying the one on local. And this includes core.extensions.

I hope that made more sense to you than it did to me when I took it all in. I believe I have a full workflow now that's robust enough to handle anything I can throw at it, and I hope by sharing it I will have saved you some time!

Postscript

I have an updated recommendation. In addition to making configuration splits for local, dev, and prod, I recommend breaking some configurations into different splits for different sets of modules that do things which would be enabled on different environments. For instance, you could make make settings that just included memcache, or the suite of modules that controlled purging. This would allow one to enable memcache on one's local, but allow a colleague to turn it off on their local. Then, one would enable multiple config splits in your settings.{env}.php (yes you can do that). Like this:

$config['config_split.config_split.memcache']['status'] = TRUE;
$config['config_split.config_split.local']['status'] = TRUE;

About the Author

Hi. My name is Jeremiah John. I'm a sf/f writer and activist.

I just completed a dystopian science fiction novel. I run a website which I created that connects farms with churches, mosques, and synagogues to buy fresh vegetables directly and distribute them on a sliding scale to those in need.

In 2003, I spent six months in prison for civil disobedience while working to close the School of the Americas, converting to Christianity, as one does, while I was in the clink.