Episode 188: Requirements in Agile Projects
01:00:01 Share

This is Software Engineering Radio, the podcast for professional developers. On the web at se-radio.net. SE Radio brings you relevant and detailed discussions of software engineering topics at least once a month. SE Radio is brought to you by IEEE Software Magazine. Online at computer.org slash software.

So, Suzanne James, welcome to IEEE Software and SE Radio. Thank you. Thank you for having us. And how about first of all introducing yourselves to the listeners? Well, my name's Suzanne Robertson, and I'm a principal of the Atlantic Systems Guild. It's an organisation that's been in existence for how long, James? 27 years, I think now. And there are six of us in it. And one of my partners, in fact, is James, who's sitting next to me.

And as you've no doubt gathered, I'm also a principal of the Atlantic Systems Guild. I hope we can portray some of that as we go through the interview. Excellent. Well, what we want to talk today is looking at the intersection between Agile, which has been around now for well over a decade, and its intersection with requirements.

So I thought I'd start asking about when in your requirements work both of you became familiar or aware of Agile projects and its potential impact on requirements work. It was about, actually about 10 years ago. Wasn't it when Kent Beck wrote the book about extreme programming? I think it was a little earlier than that. At least I think Kent Beck wrote the book a little earlier than that, but without...

checking let's say 10 years. Okay well I think probably it came into the mainstream you know people he wrote very well he had some very good ideas and it captured a lot of people's attention and they said oh great this means that we can just work in a small group and we can just build things and actually one of the big mistakes I think that people made not that Beck really intended them to do this

But one of the big mistakes that people made was to say, oh, well, if we work closely with another person, we don't really need to do any requirements work because we're there and we can build software. It's very software-focused. And those are my earliest memories. What about you? I think my earliest memories are reading Extreme Programming, but also the, I don't want to use the word hysteria, but the...

extreme zealotry that surrounded the introduction of agile techniques. This has been very much a facet of software engineering that as soon as some new paradigm comes along, everybody, not everybody, but a lot of people leap onto the bandwagon and throw everything away.

We don't do what scientists do, which is stand on the shoulders of giants. As an industry, we tend to throw everything away and start again, and this happens. Suzanne and I have seen it happen four or five times in our working lifetime. The problem with agile...

and I'm talking here about big A agile following a method, whether it's XP or Scrum or Crystal or any one of the other flavors of it, is that the baby got thrown out with the bathwater. The problem was seen as requirements were all wrong, that we had to have this giant requirements document that had to be written and complete and then thrown over the transom to the development team.

And this was taking a long time for anything to happen. And requirements were pointed out as being the villain of the piece. And so the early Agilists threw out requirements. And I think that was a huge mistake. I think we're going to come back to that in a big way later on. So my early, you know, if you're asking my early...

responses to Agile was that this is living dangerously because we've gotten rid of something that's very valuable whereas really there's room for two things here. I think that would be my... Did you find that your clients were expressing that view or were they taking a more pragmatic middle road once they started to move to Agile? Well the clients were either confused or

and following some kind of a step-by-step method, or being driven by, sometimes by managers who had very little knowledge about what Agile was about. It was just, if you're Agile, that means that you can get things done much more quickly. And consequently, a lot of the teams that I was working with and have been working with just get an awful lot of pressure to be Agile, and really that translates to get this done more quickly.

And I think the more intelligent views, more intelligent approaches have been when people have said, well, hang on, what does it actually mean? And why is it a good idea? Because it is a good idea. It's a terrific idea. So what is it? And you then, you know, you make the abstraction and you think, well, what it really is, is you want to produce value, but

and you want to produce it because we build systems for people and systems, I'm talking about them in the wide, not just software but systems. And we want to be able to produce this value and we want to be able to do it quickly and we want to be able to turn on a sixpence when businesses change and they change more and more often.

And I think the people who really understand what Agile is all about understand that. They don't say being Agile means you do Scrum or being Agile means you don't do requirements or whatever it is that they pick up from the patchwork quilt. So it's understanding the motivation, the rationale for Agile rather than the mechanics of it. Oh, absolutely. I think that's my point about Big A Agile and Little A Agile. We're very much in favour of Little A Agile.

I think if you follow too slavishly a method, it can lead you down the wrong road no matter what the method is. And that was my experience with clients who were early adopters of it. They felt that if they simply did everything that the method, big A agile method, said to do, that their problems would be over.

It turns out that doesn't work. It doesn't work that way. We can't work following a slavish method, but if we're flexible and not rigid, in other words, little a agile, then it does bring benefits. Well, if people understand what little a agile is, then they can pick the techniques and

that suit particular situations. Like if they say, okay, what we're really trying to do is to understand the problem well enough as quickly as we can to decide what to do that will give people the most value. Sometimes that will mean using a particular technique or building a particular type of model. Sometimes it'll mean going through different sorts of cycles, using prototypes, using simulations, all the things.

that we know about as being useful techniques and approaches. But the problem is that because it was new and because it was presented as a technique or an approach, that's how people got to know about it. So they thought about it as a technique rather than as what's behind it. And that's what we were meaning by little a, agile.

Yes, most of my clients now are moving away from big A agile and the little a agile when they're getting more value out of better thinking as opposed to... So how does little a agile manifest itself in projects today? By the adoption of things outside of the original big A agile camp. For example, something that is near and dear to our hearts is

systemic thinking. This is not mentioned in most of the textbooks, but it's a very important part of what we do. Innovation, for example, is not mentioned anywhere that I've been able to find, but nevertheless is absolutely vital if we're going to be building useful things. And I think the... I don't know who first said this, but

And to paraphrase it, you can build something any way you want. You can build software any way you want, but if it doesn't do something useful, then by definition it's absolutely useless. And something that in the early adoption of Agile

The emphasis was on doing things quickly. Well, I'm also going to say you can build something as quickly as you want to, but if it doesn't do what's needed to be done, it's absolutely useless. And so I think clients or customers and software builders are now coming to the realization that,

It's better to take a little bit longer to do something and get it right as opposed to build something totally useless on time, on budget. Yeah, but I think another thing that you see when people are really coming to grips with Little A Agile is that they're not driven by any rigid process, not at all. What they're looking for is to come to grips with a problem as quickly as they can

so that they can make some decisions about which bits to do what with, which bits to go further with. And they might, it might be a team who work in a particular way that they like, or they might be experimenting with something else. But it's not driven by any process or any rigid process. I think one of the, you often hear about agile is that

Either the stakeholders really can't express their requirements, or by the time they've been able to express them, the requirements have changed. We're living in such a dynamic world that it's almost pointless to document the requirements in any way that we might consider traditional. What's your view on that? I think requirements change much more slowly than people say they do.

The problem is that the requirements are never gathered correctly in the first instance, and so when they're presented back to the users, the users are saying, no, that's not what we meant, and that's taken to be a changing requirement. Well, it's not a changing requirement. The basic business need is still there. It just hasn't been expressed properly. One of the things that I noticed about early projects

big A agile adoption was the emphasis was on building some. Let's go and build some software. And even this macho stuff about having iteration times timed in hours, if not minutes, in order to keep on producing software, keep on producing this, keep on producing stuff. This is producing a solution, but it's not necessarily a solution to the right problem.

just because we deliver up a piece of software does not mean you're delivering value. And this almost chanting of a mantra that went on about constant delivery of customer value is

Software is not necessarily valuable. If it doesn't do what's needed, then it's useless. It is not valuable at all. And so I'm seeing more and more now the stepping back from that constant delivery of software into getting a better understanding of what the real problem is. And I think that's extremely healthy. Well, also, the other thing that you mentioned, the D word, documentation. Mm-hmm.

I think there's a lot of misunderstanding about that as well. I think that people think about if you do requirements, it means you've got to produce a lot of formally written documentation in large documents, blah, blah, blah. But it's not that at all. What it is is you want to leave a trail. And if you're working in a co-located environment...

there's nothing to stop you leaving your trail on the walls and on flip charts. We've got video cameras, we've got cameras all over the place, everybody's got them. We've got all sorts of technology that we can use to free us from having to sit down and say, well, now it's time to do the documentation. I mean, you should only do a translation process

into another form if you need to do that translation in order for whoever it is you're trying to communicate with to understand it. So if you're outsourcing or something like that, you probably need to spend more of your budget on translation, but you shouldn't do it if you don't need to. So the reported demise of more traditional requirements techniques is premature? Well, I don't know because I'm wondering what you mean by more traditional requirements techniques.

documenting requirements in a way that makes them measurable, that you have got a specification that can be analysed for various properties, completeness and correctness. But I think, you see, that you could do that in a co-located team with white walls. I mean, I know that's extreme, but you can do it. The key thing, and this is a big key, is...

to tell the difference between the form and the content and the content is what you're trying to communicate. Now if you've got a team, for example, and they are talking to each other about requirements, let's say, and they say, "Well, we've got 125 requirements." If they haven't got a shared mental model of what a requirement is,

they're just going to lose the meaning. So you've got to have some kind of shared mental model and it doesn't have to be imposed by anybody else as long as that team understands that. If you've got that, then you can trace the requirements, you can make them measurable, you can say whether you've got this bit of a requirement, whether you've got a rationale, whether you haven't. It just might be in a different form. So Agile brings us new forms.

of requirement. The content remains, but the form changes. Absolutely. I think that requirements now, we're accepting that some requirements can be verbally communicated, that they can be sketched and scribbled on the back of story cards and so forth. There's nothing wrong with that at all. I think

It comes back partly to what are you calling a requirement, and when you say traditional requirements, if you think about traditional requirements as being the business analyst goes, talks to the customers, writes down whatever they say, translates that into some form to give the developers, yeah, that's been gone for quite some time.

If you talk about requirements in terms of, let me call this little r requirements, in terms of actually finding out what the real problem is and having some way of communicating that, and it doesn't matter whether it's written or not, then that's the way we see requirements going today. There are some needs for, you mentioned documentation a moment ago, there is a need for documentation still.

What we have learned from the agile methods, that documenting what a piece of software does is completely useless because the code is the best documenter of functionality. However, it's not important to document that simply because we can read the code to find out what it's doing. Documenting why something exists is very important.

Because in five years' time, when this software still lives, and let's face it, software is living longer and longer and longer, I sometimes do a brief survey in my requirements courses and find that there is...

belonging to organisations represented in the room that is older than the students represented in the room. So knowing why something exists is important. So that kind of documentation has not gone away, or at least the...

Companies that were developing software without keeping that kind of documentation are realising they do need it after all. The rationale behind something, which is why we have it on our snow cards to always document the reason behind something. And I think it comes back to when you talk about requirements or we were talking earlier about requirements changing, if you do discover the rationale for a requirement,

or even the rationale for a use case, or the rationale for a whole piece of work, you find that that concentrates everybody's mind on exactly what the problem is. And you find the problem doesn't change, but proposed solutions to that problem do change. And so rather than demise, I'm going to say no, requirements work is change.

into something different with an emphasis on rationale, reasoning behind things, with an emphasis on discovering the real work as opposed to the emphasis being on traditionally writing the large document. The large document, yeah.

Mind you, having said all that, there is still a need for the large document if you're outsourcing and let's face it, most organisations today are outsourcing. You need to write the complete specification because when you send that off to India or Russia or Indonesia or wherever you're sending it to, they do need a complete specification. Very few outsourcers are going to work iteratively with you.

I can see how that can work for functionality, functional requirements. What about quality requirements? Usability, reliability? I've seen them scribbled on the backs of user story cards and so on. Do you think those have to be documented in different ways in Agile? I do, yes. Because the qualitative requirements, what we call the non-functional requirements, quite often exist across several business events, use cases, events,

across a number of different transactions, whatever partitioning you want to use. So attaching them to one story card is possibly not the best thing to do, simply because if there's another similar story, they may not pick up the same non-functional requirement. So it's almost as if the...

qualitative requirements have to be separately documented because they are an overarching

of the whole development effort as opposed to one particular part of it. Yes, but they also need to be connected to the relevant bits of functionality and they also need to be connected to the people who really understand that particular type of non-functional requirement. So it's like now you've got the thread that goes from the functionality to the non-functional requirements to the particular stakeholders who are the experts.

And we're seeing this more and more, that because our systems can do many more things and can support many more non-functional requirements, we're seeing more and more internal consulting stakeholders, people who are usability experts, who just focus on that type of requirement and they're experts in them. And if there is some kind of formal way of writing those things down or...

making them available to people anyway. That means that you then start to open the door to reuse, to really to open the door to reuse, because you don't have to go and discover those usability requirements and write their measurements down every single time. But that doesn't happen unless you've actually built up that sociology in the organisation. Okay, so...

Taking Agile with a small a, how do you think your consulting and requirements practices have changed over the last decade? How have you modified Valere or changed what you teach or train people? Well, now, because people are becoming more aware of a wider picture. Not enough, but they are becoming... They're starting to realise that we have to be systems thinkers.

Because of that, we can actually get them to start thinking about the requirements really at the beginning of doing a piece of work. And I really mean at the beginning, before you actually start to say, okay, let's go and find out all the atomic requirements. Why not identify just that really simple model that we've used, the scope, the stakeholders and the goals. If you can make that visible...

as the highest level requirements, but you're actually making them measurable. And at that point, identify the pieces well enough to start prioritizing. And this is where the A, the little A comes in. Because the earlier you can start prioritizing the pieces...

the more agile you're able to be because you can say, oh, well, look, we've done this quick analysis. We can identify the pieces and we've got 50 pieces and we're going to go through a quick prioritization, whereabouts is the major value, where are the difficulties, risks, costs,

But you can do this quite quickly. Would this be on sketched requirements, requirements that are not perfectly formed as we may expect them to be in traditional ways? It's connected back to the functional side of it. It's connected back to the scope. The non-functional side is connected back to the stakeholders. So even though you don't know everything downwards in the detail...

You do know what you're talking about at the high level. In other words, high level doesn't mean I don't really know what I'm talking about and maybe I'll find out. High level means we understand this... We understand what we think it is now so that we'll know if it changes. And that means you can then... You can be very agile. You can say, OK, I'm going to work on this piece and we're going to deliver something or we're going to work on this piece because we think it's the riskiest thing and if we can't do this, we may as well...

stop doing the project. It's those things that people can use to be much more agile. I think the changes I've seen is that the big A agile folks are coming much more around to looking at

very much at the problem space and this is where if you want to talk about traditional requirements and traditional agile come together it's an investigation of the problem space to find out what is it we're really trying to do here and what we're trying to do is not to build a piece of software what we're trying to do is improve a business and there's a subtle very important and very important difference between them between the two of them uh the other one is

Suzanne mentioned prioritisation of looking at the problem space, cutting it up any way you like, but carving it up into pieces and then prioritising the pieces because a lot of what we do is not important. A lot of what has been done is not important. We've built software to carry out

functionality that's only used once every five years or isn't that important doesn't happen very often adds very little value to the owner of the software and so prioritization I think is

causing people to think about much more closely about what they're doing and not develop as much software but only develop for the important things. How little can we actually get away with here? That's interesting. It's very important. You've mentioned systemic thinking.

To me, systemic thinking involves some kind of modelling or walking through something. But it involves artefacts. These sorts of models and artefacts aren't typically things you associate with big A agile. They tend to eschew that kind of modelling and they see it as documentation.

Do you use those sorts of techniques at all in your approaches? Oh, absolutely. Very much. Absolutely. Systemic thinking is hugely important because it doesn't matter what your method is. Saying you can't build a diagram is rubbish simply because systemic thinking is difficult enough if you're trying to do it all inside your head.

or on the back of a card, it's not going to work. Typically you do need some kind of model just to see the effect of what you're doing. And building a piece of software, so let me just back up a moment here. I was talking just a little while ago about the problem space, investigating the problem space. This is something we're seeing more and more of, people thinking about the work area that they're trying to improve and not just the software.

The early big A agile went straight into a piece of software and had no idea really whether it would solve the right problem because they're just looking at this very narrow focus. Well, the focus is broadening. We're looking at a piece of work, but then systemic thinking says you've got to look at that piece of work. If we change that piece of work, what happens to the rest of the organization that it's going into? If it's a simple, simplistic approach,

A piece of software like a game, for example, you don't care about what happens around that because the person is going to buy the game or not. What else they do in their life you're not concerned with. If you go and put in a new inventory control system into a car manufacturer, you're very concerned about what effect that has on manufacturing, on finance, on personnel, on everything else.

And if we don't think about all these other things, then we've got the potential for catastrophe. But that points to something that I've seen people are more and more ready now to...

go back to things that have proved to be useful in the past and connect them to this idea of being iterative, being agile, being able to look at things in hierarchies just as good system thinkers can do. So, for example, people are now saying, well, process models work in certain cases, but

Data models work, state models work, systems flow, dynamics models work, but they're also starting to understand how you can connect those so that you're being consistent.

And you can choose which ones to use in what situations, but you can connect them into the overall picture of the project that you're doing. And that harks back to something I said earlier where people can choose the techniques...

the procedures, the visualisations that are most appropriate to what they're doing. But it does rely on a really important thing about a systems thinking skill, which is, I mentioned earlier, being able to distinguish between what is there because of a form and what's there because of a content. Not that form's bad, but just to be able to understand which is which. So do you see your role in part nowadays as helping people

a project choose the right techniques and tools to use in a sort of big pick-and-mix agile shop. There's a lot of choice out there. You may be overwhelmed by the choice. Is that a role that you see for yourselves? Oh, we do. We do that quite a bit with people. Obviously, you've got to learn a little bit about that first high-level analysis. And then from there, because we've seen so many people

approaches used we can say to people why don't you start with a data model for example because this is a very data heavy problem and you're going to get much more insight more quickly if you start with data well these people might never have

have done that before because they were used to starting with worrying about processes or functions or whatever they were used to starting. So it's trying to help people to free themselves from this is the way we always do it. But also freeing teams from believing that the model or the technique is the purpose of it. The purpose is to do some work, to improve the owner's business in some way.

Getting into wars about techniques or diagrams and so forth, a lot of what I find myself doing is saying to people, you can build a model of data any way you like, and if three different people are using three notations, that's fine, because essentially they're all talking about the same thing.

And there's no one model that's going to be notably superior to another. In fact, you can actually do it in a tabular form if you want to. It doesn't really matter. But getting people to realise that is hard if they haven't had a lot of experience because it does rely on having done...

a fair amount of project work so that you can understand these different abstractions. We've been focusing a lot on helping people to become better abstractors because I think everybody can be better at abstraction, it's just that quite often you're not taught that way.

You're taught a particular way to do it rather than what's behind it. And even if you were taught, you need a little bit more experience before you can say, oh, this is really the same as this. Oh, this will work better now. So we've been developing some models. I mean, our brown cow model is very useful.

as a way of helping people to see you can make, there are four different viewpoints that you can take of anything that are useful. Traditionally in business analysis they said as is and to be. Well, that's not really enough. You need the as is and how it's done

and the to be and how it'll be but you also need what's behind it and that's really looking at the the purpose and the essence and all that so it's models and things like that that we've helped people to use that as a starting point to help them to be more agile because if you know that you can take four different useful viewpoints you can take lots more but four useful ones

then it doesn't matter where you start. And that's what makes people agile. So abstraction is one of those incredibly useful abilities to be able to see something independently of its implementation. In other words, the underlying reason behind it. We often refer to this as the essence of the problem. And once you can see the essence, see the reason behind something, it does...

help enormously to come up with better implementations. My earlier criticism of Big A Agile rushing into an implementation was that they were not looking at the underlying problem, in other words, were not taking any abstractions at all and simply producing a piece of software. To complain about changing requirements is by not looking at the abstraction, you build a piece of software, it's probably not going to meet the real need.

And so when a change is asked for that piece of software, that's taken to be a change in requirement. Well, not at all. The original requirement is the same. And if people take a little bit of time to abstract away from implementations, then...

you can get to see the real problem. And I think that helps enormously. But you see, if you're really agile, supposing that you're trying to understand a requirement and you're having trouble and people are having trouble telling you all, you just don't get it.

then it's an intelligent thing to do to build some kind of simulation, to build a prototype, to use that as a way of trying to figure out what is it you're really trying to do. And that's really, if you can do that sort of thing, then you're really being agile. Are you beginning to see clients...

build solution abstractions, modelling architectures now, because that was something that seemed to disappear in the big A agile in the early days, moving straight from user stories to code. Where was the architecture? Is that reappearing in projects? It's very, very slowly. For a reason I cannot explain, architecture is unpopular.

It's one of those necessary things that nobody wants to talk about. And so architecture, yes, it is reappearing and people are understanding the need for it and so on. But again, architecture is not doing itself any favour with things like TOGAF 9 and only five people on the planet can understand it properly. So it's...

I'm exaggerating here, I hasten to add, but wrapping it up in this arcane kind of presentation is not doing it any favours. So for that reason, it's unpopular. Okay. Let's look at innovation. You mentioned that as one of the drivers, if you like, for projects nowadays and why we're developing software. How do you think innovation sits with Agile? I can see how the proponents of Agile could argue it's responsive and dynamic.

What are your experiences? We were actually arguing about this earlier on today. I'll present my version, which is no doubt the correct version of this. And then I'll argue with you. The most valuable innovations...

come early in the project and the most valuable innovations of those are the ones that affect the problem space as a whole. Now I'll hasten to add here innovation is not coming up with some new technological gizmo with a touch sensitive glass screen or a new iPad or anything else like that. Innovation is fresh thinking, thinking differently about the problem area.

And the most valuable innovations are the ones where the whole problem is changed because of some innovative, fresh thinking. One example that springs to mind simply because I'm doing it at the moment is First Direct Bank here in the UK. First Direct Bank, brand new bank, or was a brand new bank about five years ago.

They started by saying, let's look at the whole problem of banking and what is it that people like and what is it people don't like. So rather than coming at it from a traditional banking perspective, they said, we want to look at banking from the point of view of a customer, which is really quite an innovation when you think about it, because most banks look at banking from the point of view of the bank. And

FirstDirect came up with a completely branch free, paper free way of banking. It's all done on the internet or over the telephone and importantly in order to make this work because anybody can implement that, in order to make this work their fresh thinking or innovative thinking was

about the people they hired for the bank and they found them all in the service industry. If you'd worked in traditional banking, you're not able to work for First Direct. You had to come from an airline or a hotel or something to do with customer service and that's what they were selling. They said banking is not about money, it's about a customer service. And when you think about it, that's perfectly correct. But the concept, Tom, we had earlier on was...

You said, ah, yes, well, you can't... Being agile doesn't mean you're going to be innovative. Well, I maintain that if you are little a agile...

You are developing and applying skills that make it possible for you to say, okay, this is where we are now so that other people, quickly, quickly is the key word here, so that other people can understand what you're talking about. And then if you're really agile, you say, we're not going any further. We're going to question all these things.

We're going to question who's here. Ah, yes, but couldn't it be this? Who are we going to hire to do this? Why are we really doing it in the first place? Now, you've got to have some kind of starting point because if people just brainstorm like mad, what ends up is they have lots of ideas that float around and

That's that. But if you're really agile, you can get a starting point quickly, something that you can build on, disagree with, replace. And that's why I maintain that little a, agile, makes you, if you really, really have got it, it will make you more innovative. Well, if you think about big A, agile, and a time-boxed sprint...

one iteration, it doesn't really leave you time for innovation because you've committed to building a piece of software by a given date. That's what you've got to do. That was the core of our disagreement because you were talking about big A is not innovative and I was talking about little a is innovative, but we were just saying agile. Consequently, we had to talk it through.

Let me try and mediate the domestic dispute in front of me for the listeners in case they're worried about the consequences of this interview on your harmony. I think this time boxing is an interesting issue. Certainly, I know from looking at creativity and innovation, the process of incubation, mulling over ideas, thinking things through as possible, and that doesn't often fit well with the need to be doing something and being seen to do something.

Well, innovation is not a rigid process. It's one of those bizarre things that we don't know how long it takes to do it. We don't know how long ideas sit around before they come to fruition. The iPad, one of the most successful consumer devices in recent history, took quite a few years.

because it was kicked around, it changed, it morphed from one thing to the other. Some of the ideas from the iPad were put into the iPhone that was developed first, and then later the iPad itself came along. These things can't be timed. Now, in terms of value, the iPad is staggeringly valuable to Apple.

It's also incidentally something that nobody knew they wanted. And this is part of just sort of take a little diversion here to, let me say, big R requirements. People don't know what they want. Nobody listening to this, I'm willing to bet nobody listening to this broadcast knew they wanted an iPad before they saw one.

They'd been asked before they knew of its existence, do you need a tablet-sized device to carry around and manipulate purely with your fingers, no keyboard? The answer would have been no. But you're talking about an invention. Well, that's an invention, yes. As opposed to an innovation. I'm using it as an illustration of we don't know what we want. And so relying upon...

traditional requirements gathering techniques of sitting down and saying to the customers, sitting down and saying to users, what do you want? It doesn't work. Similarly, in big A agile, the role of the product owner doesn't work because one person from the business cannot possibly know what is needed. He or she may know

vaguely what they think might be a solution but it's proven to be flawed. Most clients are moving away from having a single product owner simply because we don't know what we want is coming true. Making use of looking at the larger picture within the problem. Are you aware of any examples of agile projects that have led to innovations within that business or more widely?

that have led to innovations? Yes, an Agile project that's produced an innovative outcome. It's such an obvious question I felt I should ask it.

I think the silence coming from the side of the table says no we haven't. Well I think we would invite listeners to email or social media their comments in on that one. By all means and I know that... In the small you can see it but I can't isolate any big thing. In the small where...

Where someone says, oh, could you really do that? I mean, it's not considered to be an innovation, but it is an innovation because it wouldn't have happened otherwise. Something even more convenient. Something that makes it possible for somebody to do more work. I think the problem with innovation is we always expect it to be a bolt from the blue or from heaven or wherever bolts come from. You say bolts. Jamaica, I think, is where they come from. Okay, so...

Let's move on looking more broadly now. There's been a lot of association with BegA Agile around certainly use of some tools and techniques, Kanban, user story cards and so on. But if you look more widely in software engineering, you're seeing a focus on governance rather than management.

focus on the softer people skills that are important. I focus on politics as well nowadays, understanding the role of politics in software engineering. Do you think these are important in agile projects?

I think they're important in all projects. I'm not quite sure what you're meaning by politics. Do you mean government politics or internal office politics? I was thinking more of internal office politics. The role of politics in the target domain, if you're seeking to change a business, there is inevitably some...

power relationship which will be enacted as a political action. Yes, I think this is leading business analysts in particular to be better politicians because having something accepted is the most difficult thing. It's not hard to come up with innovations. It's

Selling the innovation within the organisation and having people accept it is by far the most difficult part of it. I'm just thinking about some of my clients where they have said, OK, we want to be agile, and they're really trying to be small-a agile. They're really, really making an effort. The problem is that sometimes the organisations are structured in silos.

But the business analysts have got to work across those silos. And in order to do that, they've got to be able to negotiate because the different people in different silos have got very, very different purposes. So once again, the business analysts have really got to get better and better at the psychology and the politics and all these things that...

naive business analysts don't realize because they haven't been trained to do that. They think that writing requirements down and building models and being big A agile is what you should be doing.

It's a very different skill. Well, the most important business analyst skills are not taught, unfortunately. Politics are very important. The case comes to mind at the people first section of the Royal Borough of Kensington and Chelsea, a very innovative approach to presenting what is available to people and its worthwhile use.

Looking at this, unfortunately the link to People First is right at the bottom of the Royal Boroughs site and it's hard to see without scrolling. But if you go there, you see just what a fabulous piece of work it is.

very innovative, very non-traditional local government. But they had to fight. The guys who made it happen had to really, well, when I say fight, they had to convince people across different parts of the organisation that it was okay to be as different as they have been in their interface. And you can call that politics.

you can call it selling, you can call it diplomacy, whatever you want to call it. That was what was required. And it's very necessary. Do you think the reliance of Agile on communication and collaboration...

more than writing things down, has increased the importance of those softer skills that the analyst needs? I think it's increased people's awareness that they're necessary. I think they've always been very, very important, but people haven't really been prepared to admit it or been aware of it. I think that this does make people much more aware of the need for it. Well, it brings it out in the open, doesn't it?

which is important because before the business analyst was able to hide in a cubicle and write things down, and if they weren't exact, then he or she could always point at other people. Well, now with communication, with verbal communication being so important, it's very hard to hide if you're having an open discussion, and I think that's a really good thing to come out of communication.

One of the pretty good things to come out of agile methods is its emphasis on communication. Well, it means that education programs for business analysts are more and more aware that we've got to have not just a technical side, and you said this years ago, the socio-technical view, and that we have to educate business analysts in the socio as well as the technical. Does this mean you're changing your training of analysts to give them more thinking?

Thinking on the feet skills? Because it seems to me there is more immediate... Oh, on your feet, thinking. Yes, I mean, in my business analysis course, I actually teach presentation skills because part of the politics, diplomacy, selling, whatever you want to call it, is presenting ideas to people and...

people are, an audience is much more likely to buy into an idea if it is being competently presented. If your presenter is standing there shuffling, mumbling and doing an appalling job, then the idea is thrown out along with the presenter. So all of these skills of listening, of talking, of presenting,

I even try and convince people that drawing is an important skill, that if you can draw a little bit better, it's going to help you to get your ideas across. This is a very, very hard sell. No, no, because Dan Rowan, for example, on his website, he focuses very much on drawing, and he's really convinced a lot of people that you can actually learn to draw quite well.

quite well even though you might not have an innate talent for it I've done some of his lessons and they're good anybody can draw it's just a matter of them believing they can draw in order to do it it doesn't have to be great art you know when I'm talking about being the great being another David Hockney or anything it's just getting an idea just giving people the confidence I think that they can do things it's like that and what's that wonderful Japanese Kachuka? Kachuka?

Pecha Kuka. Pecha Kuka, where you get somebody to make a presentation that's really constrained by time. 20 slides, each slide lasting 20 seconds, and they're automatically advanced. So you've got 3 minutes and 40 seconds to get whatever it is across.

So you can't be mumbling about bullet points and so forth. It's got to be a short chapter. And it really is something that people can do and they don't think they can. It's a wonderful... That's a truly an agile thought. But also just ultimate time boxing. Going back to drawing or sketching or if I can lead into visualising.

Because if you can present a diagram to somebody that visualizes your idea, it's much easier for the other person to understand it and they're much more likely to accept it. So we're moving from user stories to user storyboards? Yes.

If you like, yes, any visual graphic presentation is, I think, is important. I think user stories work very well also if they're intelligently used. It's like everything, you know, it's a really good idea providing you can connect it to everything else that's going on.

But the storyboard itself, if we go back to systemic thinking, you get a much better, broader view of what's happening if you've got the storyboard as opposed to the story card. You're much more likely to see the ramifications of anything you're doing. But it's more dimensions, isn't it? It's more dimensions and a better visualization of it.

So looking to the future, what do you see as the big challenges for Agile and requirements over the next few years? Are there new trends that are going to pose challenges? Are we not there yet and we still need to improve on what we're doing in big A or little a Agile? I think my biggest bugbear, and you've probably picked this up already, is business analysts have to become better at systems thinking so that they are

and able to take new ideas and not be driven by them. And systems thinking has been around for a hell of a long time. It's just that we haven't really joined these things together. And I think we're starting to do it now. For example, Emma Langman gave a talk at the BA conference last year. She gave one of the keynotes. And she's focused on systems thinking. And she really turned a lot of people's lights on. And John, what's his name?

John Seddon gave another talk about it. And they came from that discipline, but the business analyst said, oh, yes, this is definitely a lot to do with what we do, and it would really help us if we could do it better. Does it reside in sprint zero or sprint minus one, or does the systemic thinking happen through the sprint cycles? All the way through. All the way through. Although...

I'm going to say I think the most valuable systemic thinking happens at sprint minus one or minus two or somewhere back there because the most valuable systemic thinking, the most valuable innovation, it all comes before you start to build software. We need a marathon before all the sprints. Absolutely. Our post-Olympic theme here during this interview. Post-Olympic theme.

Something I see happening that I think is a very, very healthy thing is that the religious aspect of agile is fading. People wanting to be agile, but they're adopting more and more of traditional methods.

Systemic thinking is beginning to infiltrate into Agile. Innovation is beginning to come into it and so forth. And so I see this as being very healthy because there are lots of good things in Agile, but there are also a lot of good things that are not traditionally part of Agile, or at least not part of Big A Agile, let's say. So the...

adoption of big A agile techniques is I see it as being breaking down and people are adopting a much more pragmatic approach to developing a software so is that the future is that how you see the

the future of agile development in the next five, ten years? Well, I hope so. I mean, I say I hope so because you never know what's going to happen. And we've always had yet another paradigm. You know, Fred Brooks' silver bullet syndrome.

I'm hoping, and I am an optimist, so I'm hoping that people just start building on what we already know and don't look for another miracle which stops us getting better at what we're doing. I'm seeing slightly a less emphasis on doing things quickly. Now, this is never going to go away. The biggest problem is that we can measure time.

We have calendars, we have clocks, and so that's what we measure. What is harder to measure is effectiveness and how valuable something actually is, how valuable a change is. And it comes back to something we, I think, opened our new edition of our requirements book with. If you're going to build software, it has to be optimally valuable to its owner. And

It's this idea of value for the owning organisation, which I think is slowly but inevitably becoming part of software development. Are we really producing something which is not just a quick fix for a

current problem, but are we producing something which is genuinely valuable, that's genuinely going to make a difference within the business? In order to be valuable, it probably has to be innovative.

It's very hard to think of anything being valuable if it is not an innovation, if it is not some significant change to what's there at the moment. So that's happening. I'm seeing that happening more and more. I think seeing more and more emphasis placed not on software but on business analysis itself. And I'm just using that as a...

a word to describe not the role of the business analyst but the role of actually analysing the business and finding out what the real problem is and what are we going to do to make that business more valuable. And how to actually make sure that the thread from what we need to do to make it more valuable is coherent to all the people in the chain using whatever form is appropriate to do it in that environment.

So if we take the engineering part of software engineering, it's engineering the business, engineering the work being done, regardless of what the work is. It can be commercial work, it can be scientific, it can be academic, it can be anything at all. It's engineering the work, and the end result is usually a piece of software.

But we're thinking more of the software as almost an accidental by-product of the re-engineering or the engineering of the piece of work. It's not re-engineering, it's engineering. Exciting times ahead. Suzanne Robertson, James Robertson, thank you very much for your time. Thank you, Neil. That was very interesting. Thank you for having us.

Thanks for listening to SE Radio, an educational program brought to you by IEEE Software Magazine. For more information about the podcast, including other episodes, visit our website at se-radio.net. To support us, you can advertise SE Radio by clicking the Dig, Reddit, Delicious, or Slash That buttons on the site, or by talking about us on Facebook, Twitter, or your own blog. If you have feedback specific to an episode, please use the commenting feature on the site so that other listeners can respond to your comments as well.

This and all other episodes of SE Radio is licensed under the Creative Commons 2.5 license. Please see the website for details. Thanks again for your support.