Pages

June 11, 2015

My First Ever Workshop at Nortal TechDay 2015


Recently I did my first ever workshop at the conference (it was Nortal company inner conference, so you probably haven't heard about it). It was some sort of bugbash, where participants had to find bugs and vulnerabilities in certain application. The application was written by me using Angular.js, JavaScript, Node.js and it used REST services and DB. All the technical part was on me and documentation part on Alla Tarnovskaja – co-author of the workshop.

As we started to use Angular.js and REST services in our company, the idea was to give some insight about some potential bugs that may appear in these technologies. Participants didn't have any documentation or specification – just the application with very simple functionality and a lot of bugs with different level of complexity, so every tester could find at least one bug. Testers worked in pairs, so there was an opportunity to share experience. Pairs had about 1 hour for testing, after what I revealed known bugs and some pairs shared their bugs that I didn't mentioned about. Also in the middle of the workshop we published some tools, that we suggested to use to explore the application – it was a little hint for testers, who were out of ideas.


Alla Tarnovskaja drew beautiful pictures specially for the workshop. This bug is defeated by Nortal team.

My main fear before workshop was stability of servers – application uses web and REST servers (on Node.js) and as I didn't write them before I was scared that testers will put them down with some injections or special symbols. And they did. They put REST server down about 10 times and web server – 2 times. But during the workshop I constantly monitored logs, so I managed to put them up again in a very short time (1-2 seconds), so general downtime is not more than 2 minutes. The problem is, that all pairs used the same servers, so if one pair managed to put it down all pairs didn't have access to application anymore. I was thought about giving source code of services and application to all pairs, so they could launch them on their own machines (or on virtual machines) and don't depend on others injections, but these things usually take a lot of time in the beginning and we didn't had that time.

One idea that came up during the workshop – it will be fun to do some beautiful logs of servers and put them on big screen, so every pair during the workshop could see what is going on on server. But you can do this only if pairs use one general server, which is risky.
Also I got a lot of ideas about bugs that I can put into application from workshops in Let's Test conference.

There is 2 things that I really liked in our workshop and didn't see in other similar workshops. First one – very detailed documentation about known bugs and how to reproduce them (documentation was opened at the end of workshop), so every participant could try to reproduce them at home, after the conference. Second one – one very interesting and hard to find bug, that nobody found. It's always a little bit boring to hear about revealing the known bugs if you have already found them by your own. More interesting is to see how to find bugs that you didn't found.

Doing workshop is fun and useful – usually I start to ask myself right questions only if I have responsibilities and fears. So making workshop is way more productive for me, than participating in it.

June 8, 2015

Let's Test 2015: Final Day 3

Software Talks by Julian Harty (@julianharty)


Photo from Let's Test Conference Flickr

Security Testing talk by Jari Laakso was replaced by this one. Julian Harty talked about mobile software, customers feedback and statistics. Some facts about Google: about 75% of bugs are not being fixed (usually it's minor bugs that are closed or postponed); GoogleAnalytics doesn't know data of one particular user, it aggregates all data into one big picture; there is not so much manual testers in Google – basically they collect data from users, which helps to find problems.

One very interesting thing about mobile testing: if your mobile application connects to the internet then you should test its traffic – it shouldn't send or receive more data than needed, because users pay for each byte.

And one lesson about customers service: developers (or somebody in product developing team) should follow users comments on store page. First of all, it's good source of problems and bugs. Secondly, if there is a reported problem without an answer, users are likely to report more negative comments and reduce the rating. If developers team answers that they are dealing with reported problem users are more likely to trust.

Coders To The Left by Jan Eumann (@JanEumann) and Philip Quinn


Photo from Let's Test Conference Flickr

Workshop about how to find bugs using the source code and how to fix them. Very useful workshop for me, because 5 days earlier I did something similar in our company internal conference. So it was interesting experience to see how other people doing same stuff as I did (especially, taking into account that it was my first workshop that I ever did).

During the session we worked in groups and had to find bugs and fix them. I discovered some new functionality in DevTool for myself and had a couple of ideas how I can improve application for my own workshop.


Me working in pair with Kadri-Annagret Petersen (@kadriannagret).
Photo from Let's Test Conference Flickr

Closing keynote Detecting the Heartbleed Vulnerability by Tuomo Untinen


Photo from Let's Test Conference Flickr

Seems like the presenter was not very experienced (which was a big contrast with the whole conference), so it was hard to listen to this keynote. Especially hard to focus after so intensive days. But topic was very interesting, so I even have written some interesting points: finding heartbleed bug wasn't a luck, more like a decision; vulnerability was made in 2012 in last day of December, which is just coincidence. They wanted to put some honeypots to find out does somebody exploit this vulnerability, but it went to public too fast (by OpenSSL fault), so it didn't succeed.

Main takeaway – if you notice something new and unclear – try to understand it.


Siim Sutrop (@SiimSutrop) is asking quite interesting question – was finding heartbleed bug a luck or good testing? The answer – it was a decision.
Photo from Let's Test Conference Flickr

Summary

In the first post I have already written that it was the best conference I have ever attended at. I've met there a lot of open, smart and inspired people, who give a lot of energy and ideas. You can talk about testing there in very different aspects, for example, I even participated in small talk about testing and religion. So, if you have and opportunity to participate in some conference and don't know which one to choose – I definitely suggest to choose Let's Test. Next date is already known – May 23rd-25th 2016!


Me thinking about the conference during the final keynote.
Photo from Let's Test Conference Flickr


See posts about other days:
Let's Test 2015: Day 1
Let's Test 2015: Day 2, Exploring Web App (In)Security

June 5, 2015

Let's Test 2015: Day 2, Exploring Web App (In)Security

Testability Features by Stefan Thelenius (@StefanThelenius)


Photo from Let's Test Conference Flickr

Good talk for morning session, which gave me a lot of ideas about making testability features in my project (for example, I really like the idea to have some tool, that chooses random document from the DB). Stefan Thelenius showed us testing-application that their developers have made for testing real-application. Interesting thing, that testing-application appears thank to developers, who decided to test more (by them self, not by managers order) and found out that only setup and configurations take half a time. So developers decided to create some application that allows to spend more time on actual testing. A little bit sad, that testers still don't implement new features by them self, but asks developers for that. I think testers should strive to do their tools by them self.

But you should be very careful while implementing testability features – first of all, it shouldn't relate to production anyhow; secondly – usually testability features are not being tested (because of the lack of time), so more complex they are – more chance to get false results.

Exploring Web App (In)Security by Bill Matthews (@Bill_Matthews) and Dan Billing (@TheTestDoctor)


I am interested in security for about a year now. I remember great session of Dan Billing about New Adventures In Security Testing on previous Nordic Testing Days 2014. So, when I saw this workshop and Dan's name I immediately knew that want to join it. And my knowing didn't let me down – it was really useful and interesting full-day workshop.

Thanks to this workshop I have a long backlog of security things (test cases and tools), that I'm going to try at my work. I heard about them quite a long time ago, but now I have an idea about how actually use them.

During this workshop we had a lunch, where Baldvin Gislason Bern said interesting thing: statistics works only once, because further data begins to adjust to the metrics.

Cynefin Sensemaking Surgery by Duncan Nisbet (@DuncNisbet)


Photo from Let's Test Conference Flickr

Duncan Nisbet told us about interesting framework Cynefin (kʌnɨvɪn) and we even did some exercises, but I'm not still sure how to use it in real life. Agree, that it can add some sense in some hard situations, but its implementation is still quite vague to me. Or maybe I was still thinking about security and didn't hear important explanations.


One interesting (in some sense even philosophical) thing about transactions between different domains. There are 5 domains in Cynefin framework: Complex, Complicated, Chaotic, Obvious and something between them all. You can move through these domains, but there is one special boundary between Chaotic and Obvious – if you believe that all things are simple you can crush into chaos and it's nearly impossible to go back to Obvious domain (usually from Chaos you move to Complex). Other boundaries allow transactions.

TestLab

After all workshops open bar and TestLab again.


Photo from Let's Test Conference Flickr



See posts about other days:
Let's Test 2015: Day 1
Let's Test 2015: Final Day 3

June 2, 2015

Let's Test 2015: Day 1

Here is another post about Let's Test conference. You can read at least two more detailed opinions about it by Hannes Lindblom (@HannesLindblom) or Dan Billing (@TheTestDoctor). Also hashtag #LetsTest in Twitter is quite interesting reading.

Shortly – it's the best conference I have ever attended. Longly – it's not quite conference, but some kind of community meeting. The slogan on badge – "By Testers For Testers – Because People Matter" – is absolutely true.

I arrived at Runo a day before official start, so I also write a little bit about Day 0. Venue is very beautiful and it's a good place for knowledge and thinking. The level of all services (room, bed, food, wi-fi, number of outlets etc) was quite hight, so there were no distractions at all.

Every participant got a little bear, who suggest to test (see title on his shirt), nice notebook and badge. I liked, that every one should write their name (or whatever they want) on badge by them self, so they are all individual and unique. There were also 3 k-cards (red, green, yellow) for questions.

A lot of people also arrived on Sunday, so socialization part started immediately. Han Toan Lim (@MIndfulTester), who was doing a workshop What I Learned From Juggling, brought devil sticks and taught willings to deal with them.

Kristjan Uba (@kristjanuba), Henrik Andersson (@henkeandersson), me, Siim Sutrop (@SiimSutrop) and Han Toan Lim. Photo from Let's Test Conference Flickr.

Day 1 started with keynote There Was Not a Breach; There Was a Blog by Ben Simo (@QualityFrog).

Photo from Let's Test Conference Flickr

This is actually the same keynote as in CAST last year and you can see it on Youtube – CAST 2014 Keynote . It was slightly modified, but mainly the same. Very good keynote, especially for those, who haven't seen CAST version. Ben Simo told very interesting true story combined with educational parts.
Main takeaway – if you find security bug don't do harm and don't publish info, that somebody may use to do harm.

Equipping You For the Unexpected Challenges of Testing by Emma Armstrong (@EmmaATester)
We needed to do some exercises in pairs and here comes a challenge – Michael Bolton (@michaelbolton) decided to join our pair for couple of minutes. I was in pair with Christina Ohanian (@ctohanian) (who did a workshop A Note on Notetaking Using Mindmaps and who draws beautiful mindmaps) and we needed to test prototype according to Shneiderman's Eight Golden Rules of Interface Design. Mr Bolton a little bit chided us about not making notes while we were exploring the application and shows as example which notes he would take in this case. Also he told us, that Jonathan Bach once told him, that he doesn't usually look for bugs during the exploring, but he looks for people, who can take advantage of application, which makes it easier to find important vulnerabilities, that can be exploited (it's not a quotation, it's how I remember it).
Main takeaways from workshop itself – be aware of time, complexity, skills and external factors, and you can use at least 5 tactics to deal with them: cheat sheets, heuristics, rubber duck, visualization and practice.

A Journey Towards Self-Learning by John Stevenson (@steveo1967)

Photo from Let's Test Conference Flickr

I really liked his book The Psychology of Software #Testing and even wrote a post about it.
All participants were randomly divided into 3 groups (and I randomly got into one group with my 2 colleagues) where we did a lot of exercises using ARCS model of John Keller (Attention, Relevance, Confidence, Satisfaction). Main takeaway – if you want to learn something try to write about it or teach it. Also interesting blog post was mentioned – Oracles Are About Problems, Not Correctness by Michael Bolton.


Me explaining to Siim Sutrop my values and priorities. Photo from Let's Test Conference Flickr

A tester's Walk in the Park by Ilari Henrik Aegerter (@ilarihenrik)
Unusual format of the session – we just walked and talked about testing. I liked it, despite the fact, that I expected more philosophy from Ilari Henrik. Venue is very beautiful and walking there is a great pleasure.


In the evening there was an open bar, TestLab and socialization.

Photo from Let's Test Conference Flickr

I really liked TestLab – it was great opportunity to test with great testers. For example, I tested a little bit some planning application with Rikard Edgren, who talked about serendipity on Nordic Testing Days 2014 and managed to crash the application by some random action.



At the end of the first day I was in some kind of cultural shock. It's a magical place for tester to be.



See posts about other days:
Let's Test 2015: Day 2, Exploring Web App (In)Security
Let's Test 2015: Final Day 3