This is the first draft of a guide to upgrading the instance of Spree commerce which we use at my work.
Created Monday 17 October 2011
New versions of Spree or Spree extensions may be needed and you'll need to update the DidItBetter Spree store when that happens.
There are really two kinds of modifications you might need to deploy. The first is one that is made by us, that is to say code changes that we've developed ourselves. The other is changes made by the originators of the code. More on the latter later.
Let's start with code changes we've made ourselves. It doesn't really matter in this case whether or not the code change is to the core spree code or to an extension, because our spree instance references both the same way. The code for both the spree core as well as the extensions are referenced in the store's Gemfile. Bundler is used to pull updates to this code from the github repository(s). Copy any new assets from the public directory of the updated code to the server's public directory, then restart the server and bang, you're done. Well, that's the short version but there are actually more steps in between.
Since we're talking about our own code changes here, presumably there's some development that's been done along with some testing before we roll that out to the server. Hopefully. So you need a development environment, a way to test the code and then check in the finished product.
The spree server code for our instance does live in a github repository and can be easily checked out and run anywhere, with the exception of the database itself. This means it's easy to do a run-through and test of the upgrade before we actually touch the real server. So that means running the server in at least two places: in production where customers can reach the real server and on a development machine where the developer can try the deployment themself, first.
However, that's for staging, not for actual development, for two reasons. The first is to keep the deployment process agile. If you're doing development on the staging server and an important update is suddenly needed, it's stuck behind the work you need to finish. You don't want that, so doing development separately from staging keeps you able to deploy an update at a moment's notice. Second, the real server doesn't make it easy to do development with the core or extensions. The real server pulls all code changes from the repositories, meaning that code changes have to be pushed to the network repository and then pulled to the server, even if the code was local in the first place. For this reason, to do development you need a development server where the server and extensions are in a sandbox setup that allows all the code to be run and edited locally.
The sandbox model is a checkout of the spree source code where an instance is created in a subdirectory named "sandbox". The server can be run from that directory with the normal "rails s" command, but the difference between this and the production server is that the code is live. If run in rails' development mode, you can even change the code while the server is running and see the effects. The way this is accomplished is that the sandbox directory is an instance, and it has a Gemfile which, instead of pointing to the github repository for the spree code, instead points to the parent directory. Similarly with extensions, it points to local directories instead of repositories, usually in directories sibling to the parent with the spree code.
So let's say you've done your development on a spree extension and been testing it using the sandbox. All the tests are green and you're ready to deploy. From here there are several steps:
This cycle happens without bothering the server in production and verifies that the committed code made it through the repository and deploys in a production environment. This identifies any issues with the deployment process, as well as integration with other modules already in deployment, so it is both process and integration testing. The deployment server is usually a local checkout of the same instance of the production server, from the same repository.
Once everything is verified, the changes made to the deployment server are committed and pushed to its repository. The great thing here is that if we did have to make any changes in order to get the new code working on the deployment server, these changes are committed and pushed, and simply pulled to the production server intact. Because the deployment server is an actual copy of the production server and using the same repository, all of the work done on the deployment server is simply pulled to production intact, it does not have to be redone.
The process of actually getting the updates into the deployment server also has a couple of details.
As far as the new code goes, you have to do something to pull the updates to the server. That's what bundler (the "bundle" command) and the Gemfile are for. Since the Gemfile indicates what versions we require of certain modules, we could just run the usual "bundle install" command which got us started with it in the first place, but this won't work. "bundle install" will pull the latest acceptable versions from their repositories and resolve all of the dependencies, but if the modules have already been downloaded once it will stick with the installed versions. It is conservative and will only update the versions of items in the Gemfile which are not currently satisfied. Hold that thought.
We could also simply run the "bundle update" command. This command is not conservative at all. It will go through every module and find the latest updates for all of them from the repositories and install them all in one go. This can cause problems because there may be unknown conflicts introduced in the latest versions of modules and it's usually not wise to do these upgrades wholesale like that, unless you're feeling lucky and have time on your hands. Additionally, if you haven't committed your current Gemfile.lock, you may not be able to get back to where you were.
You could instead run "bundle update [package]", which works to just update a single package, although it also goes and updates any packages on which the updated module depends. This is a middle-ground approach, but supposedly it occasionally causes issues with updating a shared dependency to a version not supported by another top-level package. So it can cause problems as well.
Usually, the Gemfile is specified very loosely, allowing bundler to install the latest version of a module so long as it meets a basic version number. Because of this, when you run a "bundle update", there are usually lots of candidates for being updated, resulting in a lot of change. Fortunately the installed versions are tracked in the Gemfile.lock file, which needs to be under version control. When it is under version control, you can make a checkout and just run "bundle install". This will install all of the necessary dependencies in exactly the versioning specified in Gemfile.lock, so you get an exact copy of the code used to commit in the first place. This is the only way to get a reproducable instance.
This Gemfile.lock file is a fallback in case you do have a problem with an update, but you may need to do an entirely new checkout to get back to the versions specified in Gemfile.lock. It is unclear to me at this point whether you can just revert Gemfile.lock in an existing install that has newer-version gems associated with it and try a "bundle install". I don't think "bundle install" will downgrade gems this way. In any case, this doesn't seem like a good mechanism for avoiding versioning problems in the first place, but it is there.
So the challenge is to make it reliable to upgrade one package and as little else as possible. It seems that the best way to do this, while it may seem a bit unflexible, is to lock down the versions of the extensions you want to control by specifying them precisely in the Gemfile. My general approach is to lock down just the spree code as well as the extensions which I manage through my own github repositories. Any other extensions which I don't manage myself, I trust that I will update infrequently enough that I don't need to worry about their interference with any other modules. But the core spree code can break lots of stuff when it changes, so that I want locked down, as well as my own managed extensions, mostly because their releases are not as rigorously tested and managed as other modules.
You can lock these in the Gemfile by specifying specific revision numbers. However, this isn't the best method when you're loading from the repositories. Instead, you can lock them to branches or commits. I prefer commits because they are uniquely precise, if a bit unwieldy. I prefer commits over versions for the spree code because I frequently want to be running the latest committed code without necessarily waiting for a release. If you're using versions, you have to wait for a release in order for the version number to increment. Also, for my modules, I don't typically go to the trouble of grooming releases at all, so the version number may not increment for long periods of development.
If you use this Gemfile upgrade technique, you don't need the "bundle update" command at all. Instead you always use "bunde install", which checks the Gemfile for updated versioning info and only updates the specific components needed. It also handles shared dependencies correctly. Finally, it means you can use a single command for all of your installation and updating, making the syntax simpler and consistent.
Check the Gemfile for examples of how to specify the commit revisions.
To do the upgrade to the staging server, then, you update the Gemfile and run "bundle install". Then you put the assets in place
As far as assets go, since we are currently using rails 3.0, changes apart from code, such as stylesheets and javascript, still must be copied manually to the public assets directory on the deployment server and then checked in there. Since the deployment server runs its extensions as gems, without the source readily available, you usually need to copy these items from the original directory without relying on the repository. Once you have the assets in the deployment server's assets directory however, you just check them in and they will be there when pulled to the production server.
Once bundler is done and the assets are in place, run "rails s". Do your testing, then commit your changes (mostly the assets, Gemfile and Gemfile.lock) and you're done.
To push to the production server, you pull the changes from the repository, run "bundle install", restart the server and test. There is no copying of assets since you did that on the staging server and pulled them with the commit.
If something goes wrong, you can try reverting to the last server commit (including its Gemfile.lock) and running "bundle install", but I don't think this will downgrade gems. Haven't tried it.
You will then probably need to make a new copy of the server, since bundler stores the gems specific for each install. You'll get a fresh set of gems at the specified versions this way. Make the new checkout and run "bundle install". You'll also need to copy the files that aren't in the commit, namely the database configuration file config/database.yml.
Hold on, this one's twisty.
One of my managers recently informed me that when you go to our website with "sitename.com", you were redirected to "support.sitename.com", not "www.sitename.com". This wasn't the desired behavior.
I thought I could solve this using mojoPortals "Preferred Site Name" feature. The preferred site name tells the server to redirect to another URL, one which contains the full site name, when a request comes in without a hostname tacked on the domain, as above. Sounds perfect, right?
Well, not quite. First of all, kiddies, don't mess with this setting unless you know what you are doing. Unfortunately for me, I frequently think I know what I'm doing when I, in fact, do not. Twice I locked myself out of my support site by forcing it to redirect me elsewhere. There's a way to fix this mistake by logging onto the server and using SQL Management Studio to find the right field in the database and erase it. You can find the field in the mojoPortal docs, I believe thanks to me reporting it to them after the first time I messed it up.
So what happens when you use the setting? Well, if you have my setup, nothing. See, first you go into the admin interface. This has to be done on the default site in order for you to see the preferred hostname setting. So first you go to the default site, support.sitename.com in my case, and then login to the admin interface.
Now here's the rub. If I set the preferred hostname to "www.sitename.com" on the default site, I lock myself out. Why? Because now when I go either to "sitename.com" *or* "support.sitename.com", I get redirected to "www.sitename.com". Which is the desired behavior for "sitename.com" but *not* for "support.sitename.com", which you can no longer reach at all and, by the way, is *not* the default site and therefore cannot undo the setting. Why? Because you can only access that setting when you're logged into the admin interface on the *default* site.
SQL Management Studio to the rescue. Fortunately the fix takes effect immediately and doesn't require any restarting of services.
Scratch head. Maybe it'll work if I set it on the other site, the non-default one? So I try this, and nothing. Of course, if you visit the other site "www.sitename.com", there's no need to use the preferred hostname because you're already at it. If that site had another name, then this might be useful, but it doesn't in our case. Why doesn't it do anything for the "sitename.com" case? Well, that's because the preferred hostname is on the non-default site, which never sees that request. mojoPortal *always* routes the non-specific domain request to the default site. The only way to change that particular host request is to use the preferred site name on the site that handles the request, which is the default site. As we've seen, that's impossible here.
More head scratching. Clearly I'm going to have to turn to another feature or technology to get me out of this.
I didn't really consider trying to put a manual redirect into mojoPortal's redirect handler (had enough of it's model) and IIS 6 itself can only do redirects on urls that exist as files in the filesystem. No dice.
However, I was already using another tool that is more powerful and could handle arbitrary redirects: URL rewriting via IIRF 2.1.
I had IIRF already installed on the server to handle reverse proxying to a web application server. I just needed to allow the same install to work for my mojoPortal site and to write the rules for it.
I was able to succeed with this approach, but it took a bit of wrangling. The first problem was that IIRF is an ISAPI filter, and for some reason it didn't want to load on the mojoPortal site in IIS manager. This may have been an issue with some of my security software, so I turned the security off and also ended up approaching things a little bit differently, so I'm not sure which it was that solved it.
Instead of installing IIRF as a filter on the mojoPortal site, I installed it once for all of the sites at the top level, removing it from the web application site. This worked fine for the web application site when I restarted IIS, but wasn't working for mojoPortal.
I first made a basic iirf.ini for the mojoPortal site that just enabled IIRF and logging and nothing else. I was checking the iirfstatus url to get a report from the module, which worked on the web app but not mojo. What gives.
Then I remembered. The web app is on an older ASP.NET worker group because ASP.NET 4 (for mojo) was messing with its URLs. ASP.NET 4 magically mangles any extensionless URL (basically anything not ending in ".aspx" or ".html", although that's simplifying). The same was happening with the iirfstatus url, so IIRF was never seeing the request.
The article linked above talks about that in more detail and offers a couple different solutions, one by disabling extensionless URL handling (something I had been trying to avoid so I might use it in the future) and another by doing some crazy rewriting. Valuing my sanity, I chose to disable extensionless URLs through the DWORD: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ASP.NET\4.0.30319.0\EnableExtensionlessUrls = 0
Once I did that and restarted IIS, I could get to the status page and see IIRF was working on the mojoPortal site. I used the rewrite rules from the article mentioned above and it worked like a charm after that.
Whew!
If you want to change the system default monospace font for your Windows command-line, there are a few steps.
Full details are here: http://support.microsoft.com/kb/247815
Despite the usefulness of the web, an unfortunate reality is that the web is used for many, if not most, malware attacks. It used to be that to infect your system, you had to run an executable piece of software on your computer, usually by being tricked into running an email attachment or by running cracked, infected software for example.
Attackers have become a lot more sophisticated since those days. While regular virus infections such as those are still a threat, nowadays your machine can become infected just by clicking a bad link, or worse, visiting a regular, normally trustworthy site that has been compromised by the bad guys. These attacks usually get your browser to run the infected program without asking you and without any visible indication that the bad guys have succeeded. They make use of holes in your browser software, Javascript, Java, PDFs or other technologies that can be abused.
Frequently, the first step is to get your browser to do something that you have no control over and that you don't see happening, such as running Javascript or using hidden iframes, zero-pixel pictures, and other dirty tricks. Even some otherwise-legitimate sites use some of these tricks for tracking purposes so they can better track your online movements.
For this reason, I recommend using Google Chrome, the third most-popular browser on the Internet. I recommend it both because it's an excellent browser, my favorite from an ease-of-use perspective, but also because it is less directly targeted by malware authors because of its "runner-up" status (look for this to change as it becomes more popular, which I believe it will continue to do).
Chrome supports extensions, add-on pieces of software that change how it functions. If you are security conscious, ScriptNo is one such excellent add-on.
ScriptNo blocks scripts and other web nasties from running by default, showing you an icon in the toolbar whenever you visit the page. This icon is important because it tells you whether the page is trusted or not, and how many items have been blocked from running on the page. Visit one of your favorite sites and you may be shocked to learn that some are running 20 or 30 of these items on a page, not just from the site itself but from all over the web.
While "normal" sites tend to run many fewer than that, it's not out of the realm of possibility. And to be fair, many if not most of these items are innocuous or useful. For example, most page animations and popups (such as picture "lightboxes") are scripts. These are good and useful most of the time.
However, even legitimate sites include a lot of stuff I find objectionable, mostly related to tracking your movements on the web for advertising purposes. These things are blocked by ScriptNo as well unless you explicitly enable them.
And, for the most part, many many pages load and run fine without all of the added junk.
The drawback (and there's a drawback of course), is that the pages that _do_ need the added junk won't work when you first install ScriptNo. This is a problem. You go to read your email and the page loads partially, but no email shows up.
That's because all scripts are blocked by default and "web apps" like email and facebook need them to be allowed. To do this, you go back to the icon in the toolbar and click it. It then shows you a list of all the places on the Internet that this page is loading scripts from (yes, a page can load scripts from anywhere else on the Internet that it wants, and you never see that).
A word about these sites. Usually you can go ahead and "allow" the site name that you're visiting without too much further thought. However, it loads a lot of scripts from places you've never heard of. These show up in the list, but how to know whether to enable them or not?
For the most part, you shouldn't. Go ahead and enable the main site name and see if the page starts working (more on how to do this in a moment). However, if the page still doesn't work, it takes some judgment. I normally avoid anything I already know is an advertising-related site. It gets easy to figure out which these are over time because you see them *everywhere*. For the rest, if I see something with CDN in the name, those are candidates because they are content distribution networks, which feed large files like video and images. Others I address on a case-by-case basis and see if they affect the page's usability.
There are a couple important buttons by the site names when you list them from the ScriptNo icon:
Rating: If I don't know what a site is already, I click this and it shows me WOT's (Web of Trust) page on the site. Their color coding system makes it easy to weed out the untrustworthy sites. If it's green, I generally allow it.
Temp: temporarily allow the site while this browser is open. I use this to see if a site makes the page usable, and if so I allow or trust it later.
Allow: allow just that particular now and in the future.
Trust: allow all sites it that domain now and in the future. I generally use this for major sites like google.
Distrust: never allow all sites in that domain. I use this on tracking sites and anything WOT rates orange or red.
ScriptNo saves all of this information, so it learns over time and becomes less intrusive. You always need to train it on new sites whose functionality relies on scripts though, which are a good portion of sites. This is the price you pay for security, is less usability and more intrusiveness.
Having been the victim of drive-by web attacks before, it is sad to say but for me an extension like this is a necessity and I use it on all of my computers. However, I'm knowledgable about how it works, the good sites versus the bad, and I'm willing to pay the usability price. Your average user wouldn't choose to do this on their own unless all of those things were true. To each his own, and I don't recommend this tool for everyone.
However, there is one kind of user for whom I'd strongly recommend this extension, and that's anyone who handles major financial transactions on-line. Or anyone piloting unmanned drones on bombing missions, but I digress. If you run your business account online, either do it on a computer expressly dedicated for that purpose and strongly secured, or at least do it with this tool. Many bank accounts have been looted after being compromised unknown to the user.
If you're into any kind of animation, this seems like a very cool tool. It's hard to find quality Flash animation software that isn't, well, Adobe Flash. Swish is the next best thing I've been able to find and although it's much cheaper than Adobe, it's also pretty dated-looking and not the easiest software to use. I don't know how easy Giotto is yet, but the price is right, it's supposed to track the Flash UI somewhat, so tutorials should work, and thus far it looks slick and has good documentation. I'll be curious to see how it measures up.
I use Dropbox to keep my personal as well as some work items synced to all of my PCs so that I can easily pick work up where I left off while I'm on the go.
This has worked great so far, but I found myself in a situation at work where I really wanted Dropbox on another machine so I could have access to those same work files. However, I didn't want all my personal stuff that's beside it in my Dropbox to show up as well. Besides, I've got about a gig and a half I don't really want to sync.
So I found out that you can share folders between Dropbox accounts. Apparently this is just what I want. I already had the folder I wanted to share in my Dropbox, with all of the work stuff I needed.
So I opened a Dropbox account with my work email and installed it on the other machine. Then, from my Dropbox login to my personal account on their website, I was able to share the folder to my work email. From the work email's account, all I had to do was accept the invitation to share, and now that folder with just the work stuff shows up on that new machine. But it syncs to the same folder in my personal account, which shows up on my desktop and home machines. Sweet.