Upgrading Spree
This is the first draft of a guide to upgrading the instance of Spree commerce which we use at my work.
Updating Spree
Created Monday 17 October 2011
New versions of Spree or Spree extensions may be needed and you'll need to update the DidItBetter Spree store when that happens.
There are really two kinds of modifications you might need to deploy. The first is one that is made by us, that is to say code changes that we've developed ourselves. The other is changes made by the originators of the code. More on the latter later.
Let's start with code changes we've made ourselves. It doesn't really matter in this case whether or not the code change is to the core spree code or to an extension, because our spree instance references both the same way. The code for both the spree core as well as the extensions are referenced in the store's Gemfile. Bundler is used to pull updates to this code from the github repository(s). Copy any new assets from the public directory of the updated code to the server's public directory, then restart the server and bang, you're done. Well, that's the short version but there are actually more steps in between.
Since we're talking about our own code changes here, presumably there's some development that's been done along with some testing before we roll that out to the server. Hopefully. So you need a development environment, a way to test the code and then check in the finished product.
The spree server code for our instance does live in a github repository and can be easily checked out and run anywhere, with the exception of the database itself. This means it's easy to do a run-through and test of the upgrade before we actually touch the real server. So that means running the server in at least two places: in production where customers can reach the real server and on a development machine where the developer can try the deployment themself, first.
However, that's for staging, not for actual development, for two reasons. The first is to keep the deployment process agile. If you're doing development on the staging server and an important update is suddenly needed, it's stuck behind the work you need to finish. You don't want that, so doing development separately from staging keeps you able to deploy an update at a moment's notice. Second, the real server doesn't make it easy to do development with the core or extensions. The real server pulls all code changes from the repositories, meaning that code changes have to be pushed to the network repository and then pulled to the server, even if the code was local in the first place. For this reason, to do development you need a development server where the server and extensions are in a sandbox setup that allows all the code to be run and edited locally.
The sandbox model is a checkout of the spree source code where an instance is created in a subdirectory named "sandbox". The server can be run from that directory with the normal "rails s" command, but the difference between this and the production server is that the code is live. If run in rails' development mode, you can even change the code while the server is running and see the effects. The way this is accomplished is that the sandbox directory is an instance, and it has a Gemfile which, instead of pointing to the github repository for the spree code, instead points to the parent directory. Similarly with extensions, it points to local directories instead of repositories, usually in directories sibling to the parent with the spree code.
So let's say you've done your development on a spree extension and been testing it using the sandbox. All the tests are green and you're ready to deploy. From here there are several steps:
- Commit your code
- Push to the network repository
- Go to your deployment server directory
- Tell the server to update the module in question
- Run the deployment server and test
This cycle happens without bothering the server in production and verifies that the committed code made it through the repository and deploys in a production environment. This identifies any issues with the deployment process, as well as integration with other modules already in deployment, so it is both process and integration testing. The deployment server is usually a local checkout of the same instance of the production server, from the same repository.
Once everything is verified, the changes made to the deployment server are committed and pushed to its repository. The great thing here is that if we did have to make any changes in order to get the new code working on the deployment server, these changes are committed and pushed, and simply pulled to the production server intact. Because the deployment server is an actual copy of the production server and using the same repository, all of the work done on the deployment server is simply pulled to production intact, it does not have to be redone.
The process of actually getting the updates into the deployment server also has a couple of details.
As far as the new code goes, you have to do something to pull the updates to the server. That's what bundler (the "bundle" command) and the Gemfile are for. Since the Gemfile indicates what versions we require of certain modules, we could just run the usual "bundle install" command which got us started with it in the first place, but this won't work. "bundle install" will pull the latest acceptable versions from their repositories and resolve all of the dependencies, but if the modules have already been downloaded once it will stick with the installed versions. It is conservative and will only update the versions of items in the Gemfile which are not currently satisfied. Hold that thought.
We could also simply run the "bundle update" command. This command is not conservative at all. It will go through every module and find the latest updates for all of them from the repositories and install them all in one go. This can cause problems because there may be unknown conflicts introduced in the latest versions of modules and it's usually not wise to do these upgrades wholesale like that, unless you're feeling lucky and have time on your hands. Additionally, if you haven't committed your current Gemfile.lock, you may not be able to get back to where you were.
You could instead run "bundle update [package]", which works to just update a single package, although it also goes and updates any packages on which the updated module depends. This is a middle-ground approach, but supposedly it occasionally causes issues with updating a shared dependency to a version not supported by another top-level package. So it can cause problems as well.
Usually, the Gemfile is specified very loosely, allowing bundler to install the latest version of a module so long as it meets a basic version number. Because of this, when you run a "bundle update", there are usually lots of candidates for being updated, resulting in a lot of change. Fortunately the installed versions are tracked in the Gemfile.lock file, which needs to be under version control. When it is under version control, you can make a checkout and just run "bundle install". This will install all of the necessary dependencies in exactly the versioning specified in Gemfile.lock, so you get an exact copy of the code used to commit in the first place. This is the only way to get a reproducable instance.
This Gemfile.lock file is a fallback in case you do have a problem with an update, but you may need to do an entirely new checkout to get back to the versions specified in Gemfile.lock. It is unclear to me at this point whether you can just revert Gemfile.lock in an existing install that has newer-version gems associated with it and try a "bundle install". I don't think "bundle install" will downgrade gems this way. In any case, this doesn't seem like a good mechanism for avoiding versioning problems in the first place, but it is there.
So the challenge is to make it reliable to upgrade one package and as little else as possible. It seems that the best way to do this, while it may seem a bit unflexible, is to lock down the versions of the extensions you want to control by specifying them precisely in the Gemfile. My general approach is to lock down just the spree code as well as the extensions which I manage through my own github repositories. Any other extensions which I don't manage myself, I trust that I will update infrequently enough that I don't need to worry about their interference with any other modules. But the core spree code can break lots of stuff when it changes, so that I want locked down, as well as my own managed extensions, mostly because their releases are not as rigorously tested and managed as other modules.
You can lock these in the Gemfile by specifying specific revision numbers. However, this isn't the best method when you're loading from the repositories. Instead, you can lock them to branches or commits. I prefer commits because they are uniquely precise, if a bit unwieldy. I prefer commits over versions for the spree code because I frequently want to be running the latest committed code without necessarily waiting for a release. If you're using versions, you have to wait for a release in order for the version number to increment. Also, for my modules, I don't typically go to the trouble of grooming releases at all, so the version number may not increment for long periods of development.
If you use this Gemfile upgrade technique, you don't need the "bundle update" command at all. Instead you always use "bunde install", which checks the Gemfile for updated versioning info and only updates the specific components needed. It also handles shared dependencies correctly. Finally, it means you can use a single command for all of your installation and updating, making the syntax simpler and consistent.
Check the Gemfile for examples of how to specify the commit revisions.
To do the upgrade to the staging server, then, you update the Gemfile and run "bundle install". Then you put the assets in place
As far as assets go, since we are currently using rails 3.0, changes apart from code, such as stylesheets and javascript, still must be copied manually to the public assets directory on the deployment server and then checked in there. Since the deployment server runs its extensions as gems, without the source readily available, you usually need to copy these items from the original directory without relying on the repository. Once you have the assets in the deployment server's assets directory however, you just check them in and they will be there when pulled to the production server.
Once bundler is done and the assets are in place, run "rails s". Do your testing, then commit your changes (mostly the assets, Gemfile and Gemfile.lock) and you're done.
To push to the production server, you pull the changes from the repository, run "bundle install", restart the server and test. There is no copying of assets since you did that on the staging server and pulled them with the commit.
If something goes wrong, you can try reverting to the last server commit (including its Gemfile.lock) and running "bundle install", but I don't think this will downgrade gems. Haven't tried it.
You will then probably need to make a new copy of the server, since bundler stores the gems specific for each install. You'll get a fresh set of gems at the specified versions this way. Make the new checkout and run "bundle install". You'll also need to copy the files that aren't in the commit, namely the database configuration file config/database.yml.