Q&A with Jon Lane – A Remote Tester

I met Jon Lane a few years ago at UserConf in New York City. At that time, he was part of the Harvest support team. He gave a presentation about how he was leveraging various tools and technologies to create an efficient customer support system. I was impressed with his ideas and could tell he had a knack for quality.

A year or so later, we crossed paths again in San Francisco. At that point, Jon was dabbling in QA and beginning to take the lead as Harvest’s first dedicated QA person. I caught up with Jon recently to talk about testing and working remotely.

Q. What is your current role at Harvest?

A. My official title is QA team lead; there are three of us in QA and I head things up there. My role involves coordinating with the product team on new launches, writing test cases, running those test cases, writing automated test cases, and then the managerial side of things that come with leading a team.

Q. Where is your team located?

A. We’re a distributed team; one is in Chicago, another in Sofia, Bulgaria, and I live on Mayne Island, BC.

Q. As a remote worker on a distributed team, what tools & processes are in place to help you integrate with the rest of the team?

A. Well we use Trello, Slack for real-time communication and TestLodge for documenting tests. We use a little bit of Basecamp but not as much any more – we’ve mostly moved over to trello for that. We use Google hangouts for video conferences so that everyone can see each other’s faces. We also use Google docs for long form writing documents.

Q. What challenges do you encounter as a remote worker?

A. Harvest is a bit unique in that we’ve always had remote workers. The first person they hired was remote, so the company has always had that “remote” mentality.

If you’re working from home, then you’re living at work. Our main office is in New York and there’s a dozen or so people that work there, but the rest of us are distributed around the world. You feel a little bit disconnected sometimes when the New York team does fun things after hours because you don’t get to participate in those, but we try to help that by doing a couple company-wide meet-ups every year. Our next one is coming up in February – we’ll all fly into New York and get to spend a week together.

Q. What do you like about working remotely?

A. I don’t know that I could not work remotely any more to be honest. [laughs]

Having the flexibility to get up and start work in the morning, run my kids off to school, finish up work, pick up my kids from school in the afternoon, have time to cook and clean… I don’t know how I could do that if I had to commute to a job on a daily basis.

It’s afforded me a life that few people have. I live on an island with about a thousand people on it; there isn’t exactly a tech industry here. [laughs]

Q. What development methodology does Harvest use and how is it working with a remote team?

A. That’s a tricky question because we have a few different teams/products across Harvest.

We have the core Harvest app team, we have our mobile apps (Android, iPhone), desktop app, and we have Forecast, our project planning/resource allocation app that we launched about a year and a half ago – these are all separate teams and it’s really a little bit all over the board.

We’re not really strict on one methodology. Basically what we’ve done with the QA team is developed our own structure for how we work. It involves getting us the requirements, a Trello card, and then we’ll work with each team within our framework for managing things.

Q. In what capacity do you work with the development team?

A. Me personally, I spend probably half of my time working with the product teams – testing, reproducing bugs, reporting etc. The other half I spend on our automated test efforts as well as the management overhead stuff.

Test cases probably take the most time – making sure you’re writing good test cases takes time. Actually doing the tests doesn’t take all that long and chances are I’ve already run the test several times while writing the test cases, so you kind of know what to expect already.

The back and forth between me and the development team isn’t too onerous because we can link directly to one of the test cases we’ve written, this give the developers all the steps needed to reproduce the issue – so it’s pretty easy to reproduce something once it’s been found.

Q. Are you re-using test cases?

Yea, they don’t get forgotten about. Most of the test cases we write will be used for regression testing down the road. So when some features are changed or updated, we’ll be able to go back to those test cases and run them again.

Q. Do you ever find that writing test cases is almost too laborious for the value you get out of it?

A. It seemed that way starting out, but for us, test cases are almost a way of communicating with the product teams. It gives us accountability to them and shows them what we’ve done in terms of our testing.

Before, they could work on something and hand it over to QA, but that doesn’t provide the product team a lot of confidence in the test. Once you’ve documented individual test cases describing exactly what/how you’re testing, it gives the product team a higher level of confidence that we have good test coverage.

We had an incident last year where a bug slipped through into production and we were able to go back into our test cases and ask “did we have a test case for that?”, “was it giving us different results?” or “what happened there?”. In this case we found it to be something that was missed in the requirements and therefore a test case didn’t get written around it, which lead to the bug slipping through.

When I started at Harvest we didn’t have a formal QA department, people would kind of just do QA on an adhoc basis. Some people see the value of qa more than others, and writing test cases and formalizing the process really helped gain the trust and buy-in from everybody. It was probably the biggest thing that helped solidify our importance to the development team.

Q. How much automated testing do you do?

A. We went with a 3rd party solution for automated testing in 2015 and then they wound down their operations at the end of the year so we lost a good amount of work on automated testing when that happened. In 2016 we’re going to be re-evaluating that and very likely getting more into automated testing in-house using selenium. We’re already using selenium a bit and will probably be leveraging that more so in 2016.

Q. How much manual testing do you do?

A. In general I’d say 75% of our testing is manual and 25% is automated. There isn’t a lot of new features that we’re testing in an automated way before it hits production, we look at automated testing as a safety net.

Q. How do you use TestLodge?

A. We’ve actually expanded our use of TestLodge within the last year. Our general work flow is when the product team finishes work on something, we have them use the requirements section in TestLodge to give us a description of what the new functionality is, what things should happen, what things shouldn’t happen, and that kind of thing.

They (product) will then create a card in Trello pointing to the requirements in TestLodge and that sort of kicks off the testing process for us. The product team will also include any miscellaneous information we might need to know such as what staging environment the feature is in etc. We’ll then take those requirements and turn them into one or more test cases per requirement and we’ll execute the test runs from there.

Once we finish our test runs, we’ll link back to the test report in TestLodge showing any failed, skipped, and passed tests. We also use the Github/TestLodge integration because we use Github for issue tracking. So anytime a test case fails, it automatically creates an issue in Github for our development team to look at.

Q. Any advice for someone new to a testing role?

I started out in support at Harvest and happened to be more technically inclined so I was handling more of the technical questions that came in. You need to have a bit of a technical mind.

Mainly, internally we look for in-depth product knowledge. I had been here a couple years before I moved into QA so I already knew the ins and outs of where things were likely to break, the rough edges of the product etc. So by the time I made the move into QA I was pretty well versed on the product.

In general, every new hire at Harvest does about a month long support rotation answering support tickets – that really helps to build up product knowledge. Just hearing about the problems people are having with the product, clicking around and trying things for yourself.

Q. Any advice for other remote workers?

For your own sanity, try to stay connected, not only to your company but to the community as a whole. I’m on a really small island (literally) but I’m pretty close to a few major cities. I try to get lunch with other people in those cities when I can. I also attend conferences and even online webinars.

 

Jon Lane is the QA Team Lead at Harvest, a popular time tracking tool that makes tracking the time you spend on projects and tasks easy and fun. Jon’s background in technical support lead him to his current role where he built the Harvest QA team from the ground up. Follow Jon on Twitter here.