Waterfall Tester in an Agile World (thought experiment)

I've been in a couple corporate worlds that are agile... well in my option, "claims" to be agile. In my opinion, most companies that I have worked for and heard from friends are really just waterfall-light or lean or wagile. My big question for today because a group just recently acquired two QA testers to join their agile group.

This got me thinking, how does QA fit into an agile group? Our testers are expected to test after devs are done developing, so how is this different from waterfall? In full agile, QA would work with developers which I have not seen any self-claimed agile group actually do (not to say they do not exist, just saying... not sure what I am saying but seems important).

TO THE MAIN POINT... (focus, focus)... I think in the agile world, testers should be a special type of end-user. An end-user that is focused on breaking the system. When they find a bug, they report it like any end-user. Simple, clean, and feels like a nice sweet spot.

There is a little nagging feeling on this in that "isn't this testing in prod?" If we trust the agile methodology, this should not be a major concern. Small changes should have small impacts. By small change, I mean a change that has little impact. (Is that a catch-22?) What I want to say is that, there can be physical small changes that have big impacts. For example, a piece of code that is heavily coupled with almost everything, like security or audit or logging.

How does agile test those conditions? I don't think there is really any agile-method besides really falling onto a more waterfall-method. Which is also one of the reasons that I always shudder when I hear leadership say we need to go 100% agile. I may not know agile enough, but doesn't the first 30 minutes of agile training basically make that impossible unless the company really has absolutely no idea what it is doing? I mean, we have full spec'd out requirement that has been done multiple times... and you want to make that agile too?

I've been in two waterfall to agile transformations, and I've seen processes that took a couple days that are now two weeks because they need to go through more steps. Yes, agile should address this but don't like it when it becomes more waterfall so... yea. I tried... again... and again... but at some point there are more important things to fry and technically it adds more job security for my role. People look at me weird suggest something that does not really benefit me personally.

Anyways, if I ever have my only agile team and was assigned QA testers... I would have them help with writing test cases then test things in production. Bugs/issues will just go on the backlog and treated in an existing process.

Work Life: HealthCare.gov - Search Returns SQL Query-like Results

I didn't think this would actually work after reading this on a blog. What you see as the results are some sql queries. Although it looks kind of dangerous at first, these results are likely from other users entering these terms likely trying to hack into the website. Or also very likely (and hopefully), these could also be from third-party vendors who do website security checks because these are fairly common tables that you would try if you were going to guess at what the table names are.

Even if the attempts were legal, the results are very poor. I even tried one just to see if there were any results. After a few more times, I also noticed that the suggestion results changed. This may potentially just be other people trying the search terms after seeing the post thus skewing the results as well.

My interest in this is primarily on the software release and software testing side. After these spotlight issues, these are the type of things you would consider in a typical software test. I've learned my lessons in this same area over a decade ago. Even then, I've had preventions to these types of attacks. So although I cringe at the thought that something like this could come about, I am also somewhat relieved that this can be overlooked even at such a product with that level of scrutiny. 

In this case, the suggestion feature still works. The results are just not very good quality for your standard insurance seeking web visitor. Even as an software engineer, this probably wouldn't help me find what I want although it would be neat to customize my own search query. I could perhaps write something that would be specific to my needs instead of a common person's template.

But let us say that this does happen to your company, how do we prevent this from happening in the future? My quick take on this is that there is no good process to prevent something like this because it is a type of issue where once you've identified it, the issue will be fixed thus highly unlikely this will reoccur. For regression testing, you would add this to your set of search terms to look out for. I am sure that they most likely have sql injection prevention implemented in the back-end, but that does not necessarily mean that the search results would be ok. In the regression test, it would be important to make sure that the results are also good.

But how do you test result quality? At first glance, this does not appear to be something that can be automated because there are just too many ways a user could enter information. The search does not have a clear measurable method to evaluate what a good result is. These are things that search engineers battle everyday and still try to improve. Even if you find the right formulas for the common users, you may find someone trying to exploit the algorithm thus throwing the results off.

From what I can come up with at the moment, the best method is to investigate the more likely sql injection queries and have exceptions to those searches. This would not prevent other poor searches, but at least it would prevent some bad publicity while the program team figures a way to improve the algorithm.

Reference:
https://plus.google.com/111405772080232969785/posts/P1ZEXFo681C

Quote: User Software Testing

User software testing is a phase that users do not have time to do now but seem to have more time complaining about later