A good writeup by James Shore on The Decline and Fall of Agile (James Shore blog).
Software development projects are a hard thing; not necessarily because of the software, but because people are involved.
I’ve worked on the most mature (and rigid) software development process in the world, the Space Shuttle on-board software (SEI 5), and on places where no process existed (startups). And I can safely say that because not all teams and projects are created equal, not all process should be equal. I’m not religious about any specific type of process. The best process is the one that is/can be followed. The best process is the one that works for your team/people. Processes can’t be forced-fed onto developers, Agile or not. Changing or implementing development processes can kill a project, especially if it impacts the existing “culture”. It is smart to consider a phased approach to this, for example, start by exploring/adapting some of the new concepts to current projects, vs. totally diving into it and potentially putting the project in jeopardy. Know to be flexible when implementing projects and when to stop pushing for (defer) something; developers (and anyone) can easily get distracted with process-things vs. delivering a product. Time is of essence, go-to-market your first priority. Processes are overhead.
Regardless of the process used, it is about minimizing the unknowns, and this is accomplished via proper planning (as needed for the phase of the project) and visibility, and execution and control, from charter to go-to-market.
Processes are at times, OK, most of the times, rejected by developers, mainly if forced, and especially if forced at the wrong time; note: developers are not stupid and what they want is to be given clear and achievable instructions (requirements) and to build and deliver it on a reasonable time-frame. Realize that process-wise
sometimes most of the times “good enough” is OK, as long as things are delivered on time and with good quality. Be flexible and use what works for you; processes like Scrum or other are really guidelines.
I like to have initial efforts at front. I like to have initial planning that allows for “good enough estimates” on time-frames and budgets early on, and then I like to go iterative with cycles or sprints that allows for quick visibility and adjustments as needed, adapting in length based on where we are in the project, and team’s needs. Same on the design, I like to have an initial take at the architecture and design that will help with planning and projections and that provides a foundation for what is to come. Then we go iterative, incremental on lower design and implementation cycles that can be revisited, adapted, delivered, better understood.
The role of the product manager who owns/defines the product, manages priorities and the expectations and communication, who keeps the work plan up to date and manages scope, change, and risk at all times (control and execution) and who works well with the development team is key to the success of any software project. Then there is the development team, who are the stars who actually make things happen. Everyone must be accountable for deliverables and quality. And it does help to have smart, passionate and capable team and developers with
code good coding practices and who understand what (good) software engineering, design and development is all about, and a management team who understands that writing software is a dynamic endeavor.