All Blog Posts

My team, Don’t Test Me, has made excellent progress on our team project. We have essentially finished our csore project. We have implemented 25 passing test cases for five different model methods in Canvas, and the test cases are displayed in an HTML page. This past deliverable was fairly straightforward since we implemented a test superclass called Testatron. In fact, I was able to implement a set of five test cases in less than 10 minutes. I am extremely happy that we were able to design our test suite in a way that made it easier for everyone on the team to implement future test cases. This also means that our test suite should be easy for others to build on in the future if the want.

I’m also extremely pleased that everyone on my team has made an effort to properly build this project and has committed themselves to meeting each deadline on our timeline. I have no doubt that our last few deliverables will run just as smoothly!

A screenshot of our test suite

So far, our project has met all of the deliverable deadlines with ease. My team has been fantastic about working together remotely over Slack and making use of the time we have together in class to work on this project. At the beginning of this semester, none of us had any experience with Bash scripting, and only Kyle and I had any experience with Ruby on Rails applications. At this point in the project, all of us are familiar with Canvas and writing simple Bash scripts. Overall, our experiences with this project were not as difficult as expected, however, we overcame a few notable obstacles.

First, we had some difficulties installing our chosen H/FOSS project, Canvas LMS. We found that the all-in-one installation script provided in the Canvas repository was outdated and failed before it finished installing. This wasn’t too much of an issue for us because we wanted to set up the application manually at least once regardless of the all-in-one script. The biggest issue with installing Canvas was installing the required dependencies, particularly, a customized jQuery gem. Canvas uses a custom fork of jQuery 1.7.2 with a minute change. When Canvas’s asset management tool, Yarn, tried to retrieve the code from that jQuery repository, it complained about an Invalid Tar Header coming from the source of the repository. Unable to get the custom jQuery to install, we tried installing various other versions of jQuery. Eventually, we found that jQuery version 2.2.4 installed and allowed us to start a local Canvas server. Everything seemed fine until the homepage loaded, and the page seemed oddly sparse. We continued to struggle with jQuery for a while before finally deciding to submit an issue on Canvas’s GitHub page. While writing the issue, we decided to reset the jQuery version used by Yarn to get a screenshot of the issue, but that seemed to fix our problem. We’re still not entirely sure what fixed the issue we were having, but we included some steps and asset files that were only mentioned in Canvas’s Production Start Guide that seemed to fix our problem.

Our next biggest obstacle was finding a way to run specific model methods without access to the entire Canvas code base or a testing database. In Ruby on Rails, all model classes extend Active Record, a class that maps object instances to a relational database so that they can be persisted. This isn’t an issue for writing unit tests in RSpec, but it made it difficult to write unit tests that are external to Canvas. After some research, we found a gem called activerecord-tableless which allows us to essentially bypass Active Record. Once we were able to install this gem, writing and running tests became fairly simple.

At this point, I am highly satisfied with the effort that my teammates and I have put into our project. We have all advanced our knowledge of testing practices and scripting. Additionally, everyone has gone above and beyond to ensure this project succeeds and meets deadlines.

If you would like to see more about our team project, visit our team repository here.

It made me smile that Boris Beizer noted that nobody likes testing in the first few sentences of the Introduction in Software Testing Techniques. I personally dislike testing, despite knowing its necessity. In The Mythical Man Month, Brooks states that we love to program because we love making things, we love making useful things, we like solving complex problems, etc. Testing is none of those things. They may be useful, but they’re not something that people often interact with, so they’re not useful in a way that brings us much pleasure. With anything else I don’t enjoy doing, it helps to have motivation to do it, and Beizer offers a compelling argument on why testing is important.

Something I found really interesting about this introduction was the benefits and pitfalls of being a tester with knowledge of the system design. Testers with no knowledge of the software's design and intended use are more likely to break it by creating unusual, unintended test cases, which could be useful or unnecessary. The more a tester knows about the software’s design, the more efficiently you could test the software, but the tester may not think about odd use cases that would reveal a bug in the code. Beizer suggests that there should be a balance between the unintelligent testing mentality and the software designer mentality, and the separation between those two may differ depending on the project. This is something I personally need to keep in mind at my internship. My development team consists of only eight developers and one lead architect, and we are responsible for writing all of our unit tests and some systems tests. In my team, each developer is part software designer who implements a solution to a given problem, and part tester who is responsible for testing the solutions we provide.

Ultimately, we test to create low-risk software. Ideally, we implement our software in a way that is easy to test, resulting in less effort dedicated to writing tests and code with fewer bugs. That’s great, and something we should strive for, but the most frustrating thing Beizer points out about testing is that it’s impossible to prove that a system is bug-free. It’s always nice to see that all of my automated tests are passing, but it’s upsetting to think that no matter how much I test, I can never prove that my system is bug-free. To me, this makes testing even less satisfying. You can only try to determine how much testing is “enough.”

I found chapters 1-4 of The Mythical Man Month to be quite interesting, not entirely because I learned numerous new concepts (I did learn a few things), but because Brooks explained in detail several things I learned to be true about software engineering and project management through my own experiences. For instance, the central theme of the book is that “Adding manpower to a late software project makes it later” (Brooks 25), which is something I have observed in small-scale settings. Several months ago, a large feature went into the main application I work on at my internship just a few days before the latest version was sent to systems integration and customer testing. Unfortunately, the complicated logic driving that feature was incorrect, but something about the feature made it unable to be removed from the branch we were using for the next release (I would explain why, but I was not involved in this issue). Many things went wrong in the creation of this feature, but as deadlines approached, our choices were limited, so two developers stepped in to help the original developer. The new developers, who were already acquainted with the larger system, still took more than a day to become familiar with the specifications of the feature. It took several more days to actually get the feature working. In the end, the release date was delayed by about a week. This isn’t much compared to the delays of some projects, but for my team of 9 developers who release new features and fixes every 12 weeks, this was a significant delay for something seemingly small.

Another interesting thing I noticed in this reading was the observation that programmers and project managers tend to be too optimistic about the time it takes to complete a project. I am certainly guilty of this. Over the summer, I took charge over a somewhat large UI refactor of an outdated portion of the application I work on at my internship. I originally estimated the project to be finished in November 2017, including unit testing. Now that it’s almost October 2017, and having experienced several complications, I realized I was probably too optimistic. Luckily, the project is a low-priority project, so the release date can be moved back. In the future, I know to plan for the fact that I, like many other software developers, am often too optimistic.

If there were an easier way to estimate time, and realize that we are often too optimistic, more projects might not fall behind schedule. But, as Brooks explains, this is one of the difficulties of project management that is difficult to overcome. If we underestimate how long a project will take to complete, then it will either be late, or be incomplete compared to the original vision. If we overestimate the time it takes to complete a project, the resulting product may be obsolete come time to release it. He did note, however, that projects tend to follow a set timeline better if at least half of the schedule is devoted to testing since we tend to be the most optimistic about testing and creating bug-free software.

The last—actually one of the first—things I noticed about Brooks’ book was how he consistently referred to programmers, project managers, and other professionals as men. In fact, the title is Mythical Man Month, in reference to the number of men assigned to a project. This didn’t hinder my ability to read and understand Brooks’ observations, but it was certainly distracting. In fact, his only reference to non-male people in chapters 1-4 was explaining that “the bearing of a child takes nine months, no matter how many women are assigned.” This was a good example for tasks that have sequential constraints, it was bothersome that the only mention of a non-male was in reference to a woman giving birth. While this reading assignment was a worthwhile read, I felt as though it was never talking about me or people like me, despite being a software developer. In this book, women are mothers who give birth, and men are programmers, managers, surgeons, etc. It felt as though it was specifically referencing male developers, further solidifying the stereotype that programmers, and professionals in general, are men.

In 11 Predictions for the Future of Frogramming, Peter Wayner discusses 11 ways the future of programming and technology will change or remain the same. Some of them I already predicted myself, such as the Internet of Things becoming much scarier regarding security, and others came as a surprise. I agreed with some of his predictions, and was skeptical of others. Regardless of anyone’s predictions, attempting to predict the future state of programming is a worthwhile effort. After all, no company wants to be behind another when more money can be made.

One of Wayner’s predictions that surprised me the most was that data will continue to be essential in our software. To me, this prediction seemed so obvious that it was hardly worth mentioning. Until we become all-knowing beings, data is the only way we can begin to know the state of something. Even when the best data analysts end up being wrong, like when nearly every poll predicted Clinton would win the 2016 presidential election turned out to be wrong, what else do we have? All we can do to make election predictions more accurate is to do something different with the data we have. Maybe we can represent it differently, sample it in a new way, or use different algorithms on it, but in the end, all of our software relies on data. Data isn’t going anywhere anytime soon, and programmers will need to continue to deal with it as software becomes more data-intensive.

Wayner also predicted that autonomous transportation is here to stay, and programmers will no longer decide what we, but where we go as well. In his prediction, Wayner pointed out a particularly disturbing aspect of autonomous transportation: advertising in the form of diverting our journey. While advertising a local restaurant is harmless in itself, changing my route every so often to drive past some business can create traffic nightmares and alarming privacy concerns. Current traffic apps such as Google’s Waze are already creating terrible traffic on neighborhood roads according to a 2016 Washington Post article. Autonomous transportation, such as self-driving cars, take all control away from their users. With a traffic app, a user can choose to ignore the directions presented to them, and go a different way. They could even go as far as turning off location services so no app can track their location. Self-driving cars, on the other hand, will likely always require location information to accurately navigate the car, and can take the user on whatever route it maps out, unless there is some sort of override feature. I have no doubt that autonomous transportation will become more prevalent in the future, however, I believe as software engineers we must tread carefully regarding traffic and privacy.

I noticed that many of Wayner’s predictions are based on the notion that the number of devices that run some sort of software will expand, and programmers will need to adapt to the fact that software development will most likely no longer revolve around PCs. In my home alone, I have at least six devices that can connect to the internet that are not a PC. My mother has even more than that, including a small home security system and an Amazon Alexa. From my point of view, it’s easy to see that PCs are no longer the centerpiece of personal technology. As software developers and engineers, we need to be adaptive to the rapid shift away from PCs alone to PCs and everything else that runs some sort of software and their implications regarding safety, security, privacy, the environment, and so on.

Articles

• Security and Privacy Vulnerabilities of In-Car Wireless Networks: A Tire Pressure Monitoring System Case Study by Ishtiaq Rouf et al.
• SPY Car Act of 2015
• Planning for the Future by Mike Wood
• Introduction to Test Driven Development (TDD) by Scott Ambler
• Wikipedia article on The Magical Number Seven, Plus or Minus Two

Reflections

All of these articles, except the Wikipedia article on The Magical Number Seven, Plus or Minus Two, revolve around building safe, secure, and reliable software. In fact, the first recommendation for creating secure tire pressure monitoring systems in Security and Privacy Vulnerabilities of In-Car Wireless Networks was to follow basic reliable software design practices. They noted that impossible values were allowed to be displayed in the studied tire pressure monitoring systems. Had reliable software design practices been followed, some of the more obviously incorrect test cases could have been avoided. One possible reliable software design practices that could be used to design future tire pressure monitoring systems is test-driven development (TDD). The idea behind TDD is tests are written before development, and development is centered around tests, and tests must pass before development can continue. Ideally, the case where a valid tire pressure that also displays a tire pressure warning flag should have been caught before developing the system, and a test would have been written for that case in TDD.

Five years after Security and Privacy Vulnerabilities of In-Car Wireless Networks was published, the SPY Car Act was enacted. The SPY Car Act is essentially a federal regulation to ensure sensors and other computer systems in vehicles are secure. It even outlines a specific design principle to ensure vehicles remain safe in the event of an attack: “ISOLATION MEASURES.—The measures referred to in subparagraph (A) shall incorporate isolation measures to separate critical software systems from noncritical software systems.” This would ensure that if something like the tire pressure monitoring sensors were spoofed, the safety-critical systems in the car would retain their integrity. Ensuring a system is safe and secure also ties in with the notion of reliable systems. If a system is unsafe, or can easily be attacked, it has the potential to not function as specified, making it unreliable. Reliable systems must be able to fail gracefully, that way when the system is down for whatever reason, it causes as little trouble to its users as possible. Mike wood outlines multiple ways to handle failure in software systems in Preparing for the Future, and stresses that failure is not a matter of “if”, but “when.” After reading through each of the techniques, it became clear to me that regardless of what technique is implemented in a system to prevent or handle failure, those techniques are useless if the use case for the cause of failure was not foreseen. This is where, I believe, TDD is a useful software development practice for creating reliable software.

I also believe TDD supports the idea that humans have a hard time keeping more than seven—plus or minus two—objects in active memory at once. In TDD, you think of several test cases to develop code to pass those test cases, and repeat. This allows a developer to focus on test cases alone before worrying about the implementation of a system to pass those test cases. If we develop in a way that mirrors how much we can think about at one time, then the resulting system should be more reliable, safe, and secure.

Articles

• An Investigation of the Therac-25 Accidents by Nancy Leveson and Clark Turner
• After Stroke Scans, Patients Face Serious Health Risks by Walt Bogdanich
• Motor Vehicles Increasingly Vulnerable to Remote Exploits a PSA from the Federal Bureau of Investigation, the Department of Transportation and the National Highway Traffic Safety Administration
• The Role of Software in Spacecraft Accidents by Nancy Leveson
• Who Killed the Virtual Case File? by Harry Goldstein
• FBI Sentinel project is over budget and behind schedule, say IG auditors by Jeff Stein
• Years Late and Millions Over Budget, FBI's Sentinel Finally On Line by Damon Poeter
• FBI’s Sentinel System Still Not In Total Shape to Surveil by Robert N. Charette
• Textbook readings (Software Engineering by Ian Sommerville) chapters 11 through 14

Reflections

Each of the above readings revolve around a series of detrimental software failures or potential software failures due to a multitude of improper software engineering practices. Some of the incidents, such as the accidents described in The Role of Software in Spacecraft Accidents were financially costly, whereas others, such as the Therac-25 accidents, cost some patients their lives. To me, it is obvious that ethical software systems should never cause financial or physical harm to anyone or any environment. Yet, it seems to be far too common. As I read these articles, I wondered how these issues slipped through the cracks during development, and how could they have been prevented.

Leveson described the most common reasons for security and safety failures best: tight budgets, poor communication, unnecessary software complexities, reused code assumed to be safe, poor or missing specifications, poor engineering standards and practices, etc. One of the most upsetting things to read as a software engineer — and most frightening as a consumer — is that when budgets are slim, “the first things to be cut are often system safety, system engineering, mission assurance, and operations, which are assigned a low priority and assumed to be the least critical parts of the project” (Leveson 5). What’s worse, is that one a system has proven itself successful, the engineers of its descendant systems tend to assume that the new systems will function safely and securely as well. For example, the Mars Climate Orbiter used a navigation system that worked well for over 30 years, so the focus of the project was shifted to meeting budgets and deadlines instead of mission risk. Additionally, the Therac-25 radiation therapy machine failed on multiple occasions due to a software system that was assumed to be safe due to the success of the Therac-20. The assumption was misguided however, because the same software flaw was present in the Therac-20, except the Therac-20 never harmed any patients because it featured hardware safety checks that were falsely deemed unnecessary in the Therac-25. In order to avoid issues like these in the future, systems must be checked for safety, regardless of how much of that system was repurposed from an older system. In fact, for safety-critical and security-critical systems, it might be safer and more secure to rewrite new software to avoid overconfidence in a new system.

Another point that caught my interest, was the following statement from our textbook regarding security challenges on page 368: “When considering safety, you can assume that the environment in which the system is installed is not hostile. No one is trying to cause a safety-related incident.” I found this to be relevant to the FBI’s warning about modern vehicles becoming increasingly vulnerable to remote exploits. Before cars commonly featured USB ports, diagnostics ports, Bluetooth, or Wi-Fi, the main concern when designing a car was safety, and it remains the most important aspect of vehicle design today. However, as vehicles become more reliant on software, and more connections to that software are introduced, security becomes another major concern in an industry that hardly focused on security in the past. As I read the public service announcement from the FBI, I wondered if vehicle manufacturers were slow to address security concerns because it has never been an issue before. Maybe, those manufacturers assumed that, like safety concerns, nobody would deliberately attempt to hack into a car. After all, security concerns in vehicles are sometimes safety concerns as well if the attacker can gain access to the vehicle’s steering or brakes in transit. Now that the FBI has brought those concerns to light, hopefully manufacturers will begin to address those concerns as well.

Software development may often seems simple and straightforward. At the very least, it may seem as though software engineers have a decent idea of the costs of developing some software system. However, as Frederick Brooks notes in No Silver Bullet: Essence and Accidents of Software Engineering, it can quickly become a business nightmare. Unfortunately, the nature of software means that it will likely never have drastic reductions in development costs from a single advancement in technique or technology. Brooks explains how software is inherently complex because ideal software should not include repeated implementation (think DRY: Don’t Repeat Yourself, make a subroutine that can be called more than once). Additionally, software revolves around arbitrary human constructs that are subject to constant change rather than nature. Plus, software is invisible, and intangible, making it more difficult to visualize and understand. Brooks also outlines several potential “silver bullets” to make software engineering more efficient, but given that he wrote No Silver Bullet in the 1980’s, I believe examining current working software engineering solutions alongside Brook’s speculations is worthwhile.

When I think about massive software projects with daunting software engineering problems, the first company that comes to mind is Google. According to Why Google Stores Billions of Lines of Code in a Single Repository by Rachel Potvin and Josh Levenberg, 95% of the software used by its developers worldwide is stored in a single repository. At first, it sounded insane that a company with so many applications and thousands of developers working on those applications across the globe are all committing code to one massive repository. But, Google couldn’t become the giant that they are today if their developers spent most of their time sorting through thousands of merge conflicts. Google has mastered time-sharing, but not in the sense that Brooks described. Brooks claimed that in the time it takes to compile and execute a program, a developer can forget what they were working on. Therefore, as technology got better, it would take less time to compile a program, meaning developers would remember where they left off, which increases productivity. Even though nearly every program can be compiled virtually instantaneously today, time-sharing is still something to continue perfecting. As applications get larger, and more and more developers are assigned to work in a single repository, a single developer’s code base may be several commits behind the current version. Today, a developer may not forget what they were working on in the time it took to compile their code, but the application they were working on may have changed while they were working. Software such as Git and Subversion have made it possible for developers to work on a single repository at the same time, but it’s not flawless.

For instance, in Kode Vicious, a reader asked George Neville-Neil when they should give up cherry-picking changes into a branch and merge. Neville-Neil responded by saying that most developers end up in nightmarish cherry-picking situations because they rarely merge whatever code they are working against into their own code. Therefore, for moderately sized repositories, the issue of being far behind a remote code base can be fixed by getting in the habit of merging often. However, Google’s repository is far from moderately sized. Even a developer with perfect merging habits could be far behind the latest version with each commit due to the sheer size and scope of Google’s repository. Google overcame this issue by using trunk-based development coupled with high programming standards. Trunk-based development is where all developers commit to a single trunk, which is similar to the master branch in git. There are no long-running branches, meaning fewer conflicts because there is only one true code base and no confusion over which branch to merge into a local branch and vice versa. Google also implements automatic testing before a commit is merged and enforces strict code review so that bad changes are unlikely to be merged into the trunk.

In my opinion, the most interesting about Google’s time-sharing advancement, trunk-based development using Piper and CitC, is that it is still no “silver bullet” for Google’s massive repository. It’s simply a better engineering solution compared to what was available decades ago. The cost to maintain code integrity is high in trunk-based development compared to branch-based development. Also, Google has to control permissions on files that could be damaging to their repository, such as certain configuration files, making open source contributions difficult in their trunk-based development environment. In fact, Google’s Android and Chrome teams use Git versioning and are not a part of Google’s massive repository because open source collaborations are essential to those applications. Despite impressive innovations in software engineering since the time No Silver Bullet was written, it seems as though Brooks’ claim that there will never be a “silver bullet” has yet to be disproven 30 years later.

My name is Samantha Word, but I generally go by Sam. I am a senior at College of Charleston majoring in Computer Science with a minor in Mathematics and Astronomy.

I am currently a Ruby on Rails developer for MUSC. The primary application that my teammates and I work on is SPARCRequest. SPARCRequest is an open source application used by multiple unstitutions across the US. It is best described by our README:

SPARC (Services, Pricing, and Applications for Research Center) is a web based research management system that integrates research and clinical care workflows with a focus on service request catalog, billing compliance, proposal and budget development, work fulfillment data collection, invoicing and billing, and impact data through grants and publication associations.

When I am not studying or working, I love to cook. I find cooking to be immensely rewarding, especially when I perfect a recipe made from scratch. It's just as rewarding as creating a piece of software, and it provides a meaningful break from technology. Someday, I would like to be a Software Developer or Engineer with a garden full of fresh vegetables.