The client installs it. Then they try to run it….. Uh oh.
You cringe in embarrassment as you get the call:
Customer (irritated voice): “Your model doesn’t work. I thought you said you were done. This is broken.”
You (shocked voice): “Well, it works on my machine!”
Customer (annoyed voice): “…… Maybe you didn’t hear me.”
You (backpedaling voice): “Well, let’s see how we can fix it. Did the install process go okay?”
Customer (discombobulated voice): “Sure, it said it installed successfully after I clicked OK a bunch of times. There were a couple of weird messages – something about libraries. Then I opened the Modeling Studio like you showed me the other day. But when I tried to run it, nothing happens.”
You (staticky voice): “Hello? Can you hear me? I think we have a bad connection. I’m gonna have to call you back – are you free four months from next Tuesday?”
This is what we call the OOBE – the Out Of Box Experience. And in this example, it’s a bad one.
What’s an OOBE?
There is no impression like the first impression. If something goes wrong before the client even has a chance to use your brilliant, wonderful simulation masterpiece, you’ve dug yourself into a hole that can be hard to climb out of. It’s like showing up at one of those speed dating things right after eating a garlic and anchovy pizza. [2] Or like showing up to a job interview in spandex tights. [3]It is your job to make sure that the customer’s OOBE is as good as it possibly could be. If you give your customer a bad OOBE, what effect do you think this will have on the customer’s perception of you, your product, and your company? What’s a more likely thought process: a) “If I can just get it to run once, I bet everything else is absolutely perfect!” Or b) “I can’t even get it to run once – I wonder what else they did wrong?”
Think of it this way: Pretend you are sending the model to your Aunt, what would you want her experience to be like when she installs your software? (For the sake of this discussion, pretend it’s an Aunt that you like, not that creepy old aunt with all of the cats running around the house.)
Before a project deliverable goes out the door to your Aunt, or to your customer, it's time for a thorough OOBE test. This is different from conducting structured testing of the specifics of your model. The objective is to step out of the project’s intricate details, put yourself in the customer’s shoes, and assess the Big Picture. Does the model run the first time? Do the links in the Modeling Studio’s navigation panel work? Do the logos and images show up where you’d expect them to? Do all of the outputs load correctly? Can you save and restore a scenario?
Note that none of this has to do with the specific functionality of your model -- it's testing the little stuff you take for granted. You’ve been working on the project for weeks, and you’ve just come to assume that all of these little things are in place and that they just plain work. But that’s on your development computer. What about the customer’s environment?
This is where a Virtual Machine (VM) comes in. A VM is just like a computer within your computer. You set up a VM to match the customer’s expected environment, and boot it up just as if you had another physical box in your office. Then, you can install your model onto that VM so you can fairly evaluate the OOBE.
A VM test should be the final step in your project process before you deliver a release to your customer. This is not “something we might do if we have time and budget”. This is a critical part of ensuring the quality of the product you deliver to your customer.
What’s in it for me?
I hear you saying: “Customer this, customer that, blah blah blah, what do I get out of it?” Ah, you selfish simulationist. But believe me, a VM test helps your project in many ways:- You save face. You get to see what the client sees the first time they install and try to run your model. By doing this, you can anticipate any problems they might encounter – and fix them – before you get embarrassed by them. You personally get to put your own stamp of approval on the OOBE, and make sure there is nothing in the way of the client’s beaming appreciation of your glorious work of art.
- It helps you remember. By conducting a VM test, you ensure that your installation package is complete and that you haven’t forgotten any key files that are required to run the application. Ever seen what happens in the Modeling Studio if you forget to include one of the files defined as part of a scenario? It will completely reformat your hard drive. [4] Good thing you’re testing on a virtual machine!
- You ensure that your application doesn’t rely on software that’s only installed in the developer environment. We used to have this problem with DynaZip, which is used by the Scenario Manager, where if you run it on a developer’s machine, it looks like you only need the library dznet.dll in order for it to function. However, there was an additional file dzncore.dll, inconveniently placed out of the way in C:\Windows\System32, that was required. If you forgot to include this in your installation package, bam! Modeling Studio crashes. We found out about this problem via testing on a virtual machine.
- Similarly, you are checking how your own software will function within the client’s environment, which is probably different than TranSystems’ environment. Chances are that the customer’s IT Department has mandated the operating system, version of Office, and version of Java on each of the staff computers. For example, we’ve got a significant client that is still using Office 2000 (It’s 2009, by the way, and Microsoft has released 3 versions since then). You know that fancy chart you created using Excel 2007 on your machine? Guess what, it looks like a bunch of spaghetti in Excel 2000. Matching the customer environment on your virtual machine gives you the best chance of detecting these types of problems before the client finds them.
- Maybe you’ve spent weeks in the formal testing process. You’ve checked every combination of inputs and made sure that the outputs produced by your simulation are meaningful. You’ve checked boundary conditions, zero value inputs, and have flooded the system with 3x the volume expected to prove that the model does not crash. You have been an excellent Tester. You’ve done it the Right Way. Now how discouraging is it if the customer can’t even run the darn thing?
- If something goes wrong in the install process, your own development environment is not compromised. For instance, we had this problem with the infamous Scripting Runtime Engine, scrrun.dll. This is one of the core libraries that ships with some versions of Windows but is not always guaranteed to be found in a client’s environment. Sometimes, we write code that references the Scripting Runtime so that we need to include scrrun.dll in our installation packages. (Hi2u, RMS!) If you install this directly on your computer, and then you later uninstall this same package, then scrrun.dll might be removed from your computer. To coin a phrase, that can really grouse up the skizzlewits. A Virtual Machine protects you from all of this.
As I often preach, improve the quality of the process and you improve the quality of the product. A VM test, just before release, is one those process steps that’s easy to do (and easy to make excuses not to do), and there’s really no downside to doing it. So do it. You only have one chance for your product to make a first impression. And a good first impression goes a long way toward customer satisfaction.
Which, as consultants in the service industry, is what we’re all about.
So how’s your OOBE?
Next installment: How to set up a Virtual Machine on your computer, featuring a special guest author!
[1] In this context, “Who’s The Man” is a gender-neutral term that applies to all of you.
[2] Which the author has never done. But I did know this guy once…
[3] Professional wrestler job interviews excluded.
[4] It’s an Easter Egg I had Geoff put in there in 2006. Try it. I dare you.
No comments:
Post a Comment