Q-Success
Software Quality Management Consulting
Frequently Asked Questions answered by Q-Success
We have compiled a list of frequently asked questions in the areas of Software Process Improvement in general, Capability Maturity Model® and Software Quality Assurance.
Software Process Improvement (SPI) - FAQ
| Q: | How much effort do I have to put into SPI? |
| A: | This depends, of course, much on your organization, for example on its size, the type of development and how advanced your processes are already. As a rule of thumb if you want to be successful, you should not be afraid to spend 5% of the software development effort or more on SPI. 5% or more sounds like a lot, but you should know, that many companies that spent that amount could report a formidable return-on-investment later.
|
| Q: | What can I get in return for my money? |
| A: | You should really ask yourself first "what do I need an SPI program for?". Then you should define measurable goals, and base your SPI program on them. Just to give you some ideas, your goals would most likely include one or more of the following:
Studies show, that an SPI program, if set up properly, can generate a return-on-investment of 300% and more. Some organizations report well over 1000%.
|
| Q: | What is the most important precondition for a successful SPI program? |
| A: | Running a successful SPI program is not exactly a simple task and requires much more than just an assignment to somebody to do it. The most essential ingredients is management commitment. If the people managing the software organization are not convinced and cannot be convinced that SPI is important, then any investment in SPI is likely to be a waste of money. Full support (not only money) from higher management is an absolute necessity. When that is available, everything else that is necessary can be made available. |
Capability Maturity Model IntegrationSM (CMMISM) - FAQ
| Q: | What is this CMMI you keep talking about? |
| A: | CMMI is a process improvement model for software development. The basic
idea of CMMI is, that every organization has a certain capability to
produce software, in CMMI lingo a certain maturity. Depending on the
maturity, there are certain aspects of software development, that should
be addressed next. CMMI expresses the maturity in levels 1 (lowest) to 5
(highest). For each level, there are so-called key process areas (e.g.
requirements management, project planning, configuration management),
which need to be managed satisfactorily by the organization to achieve
that level. Each key process area has a set of key practices, which give a
very concrete idea of what is expected to be in place for a satisfactory
implementation.
|
| Q: | Who defined CMMI? |
| A: | CMM, the predecessor model of CMMI, was defined by the Software
Engineering Institute (SEI) in the early 90s. The initial
ideas were produced as contract work for the U.S. Department of Defense, but quickly,
the software industry began to take interest and supported this work.
The first version of the improved model CMMI was made available im 2000.
|
| Q: | What is the difference between CMM® and CMMISM? |
| A: | CMM was published 1993 and was an immediate success in terms of
acceptance by the software industry. Based on that success, a plethora of other
"Maturity Models" were defined in a gold-rush mood, by the SEI
and by others, basically extending the ideas to other areas than software
development and creating quite some confusion generally. Some of
these extensions together with elimination of weaknesses in the original
CMM were combined by the SEI into CMM Integration, CMMI. The
scope of CMMI is not only software, but multi-disciplinary development,
although one can still see the strong foundation in software.
Most things we say here about CMMI are equally true for CMM.
|
| Q: | What is so special about CMMI? |
| A: | If you expect CMMI to guide you to some highly advanced
software development methods, you will be disappointed. CMMI does not rely
on any method that was not known 20 years ago. CMMI works on another level.
It tells you, which of the many, many things you have to consider in SW
development you should concentrate on, and tells you which questions you
should ask yourself. It does not tell you how you should answer those
questions. This sounds quite simple and un-exciting, but
was obviously exactly what the software industry needed to make real progress in the
way of developing software.
|
| Q: | Are there other comparable improvement models? |
| A: | Of course there are, but none of the other models has ever gained so
much acceptance. One of the models worth mentioning is SPICE
(alias ISO 15504). Q-Success' specialization in CMMI doesn't mean
that we disapprove of other models. But CMMI is a very useful, proven model,
and even if all other things were equal, it is often an advantage to use
what most other people are using.
|
| Q: | What is the relationship between CMMI and ISO 9000? |
| A: | CMMI is a model specifically designed for development, with some focus on software development. It gives
very concrete guidance on what to do (not how to do). ISO 9000 is a
generic model, and although there are related software specific
interpretation guidelines, and although the last revision ISO 9000-2000 is
more process oriented than previous versions, we find CMMI a much more
helpful model, if your core competence is software development. Looking at
the scope each model covers, CMMI is more extensive (again specifically for
software development), but doesn't cover some of the more formal
requirements of ISO 9000. If you want to (or have to) use both CMMI and ISO
9000, we suggest concentrating on CMMI, and you will find that most of the
ISO 9000 requirements are covered, and the remaining ones can easily be
added.
|
| Q: | What is an assessment? |
| A: | Assessments are procedures to verify the compliance to a model. In a CMMI
assessment, your result will be a certain CMMI level (meaning all
requirements up to that level are in place) and a list of non-compliances
of the next level(s). The latter is a starting point for the next
improvement phase. CMMI also offers a continuous approach (see below), where maturity is not expressed in levels. CMMI assessments can be done by the organization itself,
by externals, or by a combination. Q-Success offers support for
guided self-assessments and for cooperative assessments, see our assessment page for more information.
|
| Q: | Where can I obtain the CMMI text? |
| A: | On the SEI web site you can download the CMMI
text for free. If you are confused with the
various versions of CMMI, we suggest looking at the staged representation, as this is in most cases the easier approach if you are just starting.
On our book review page, you can also find a book which contains the CMMI text.
|
| Q: | What is an average maturity level? |
| A: | Considering that there are maturity levels 1 - 5, one could expect that
an average SW organization is around level 3. This is not the case. Still, after many years of CMMI
experience in the industry, level 4 and 5 companies are very rare. The SEI
publishes a quarterly report, summarizing all SEI compliant assessments,
which shows that although improving, the average maturity level is around
2. And despite the popularity of CMMI, it is still only a small minority of
all software developing companies
who ever made a CMMI assessment. If we assume, that the companies who don't care about SPI have a low maturity,
then we can say that attaining
level 2 is a very good achievement.
|
| Q: | Why isn't there a level 0? |
| A: | That is very strange, indeed. 'Zero' would be such an appropriate label
for what is happening in a typical level 1 company. Somebody once suggested, only
half joking, to introduce also negative levels. We have seen companies that
could easily apply for level -3. Our explanation for not having level 0 is,
that the original authors of CMM were too polite to let the majority of
the software industry score zero.
|
| Q: | I think for me it's sufficient to be on CMMI level 2. We are not producing spacecrafts after all. What is your opinion? |
| A: | Given the fact, that so many companies are on level 1, it is tempting to
think that level 2 is superior to what many of your competitors do, so it
could be enough to keep them away for a while and not invest any more in
SPI. We believe that is a very superficial way of thinking. The right way of
approaching that question would be to look what is in level 3, and to
analyze if these things are needed. Most likely you will find that
managing the skills of your staff or building up the ability to learn from
past projects are quite helpful capabilities, both examples of level 3
requirements. The spacecraft argument is also not convincing. Even if you produce software for a quite simple device, you may find that a serious defect in that software can be more disastrous for your business than loosing a spacecraft is for the NASA.
|
| Q: | How fast can I reach level x? |
| A: | The SEI report on SW maturity also covers this question. It shows that
one can expect to climb one level in around two years. Although there is a
wide distribution for these figures, you would need to have good reasons to believe that
you can do it faster than an averagely successful SPI program. Bear in
mind, that the less successful companies never get in touch with SEI and are
thus not included in their figures.
|
| Q: | Should I use CMM or CMMI? |
| A: | CMMI is clearly an improvement over CMM. After all, many years of using
CMM have, of course, shown some weaknesses in the model. On the other hand,
the improvements are not terribly significant. CMM is still an excellent model. The
SEI discontinued support for CMM which doesn't really
mean a lot, as the CMM model itself is not going to vanish into thin air. What you should use depends on what you did in the past. If
you are starting a new SPI program, there is not much reason to use CMM.
The only good reason to stick with CMM would be your staff
being familiar with CMM. In that case investing again in training is
probably worthwile only if you benefit from other advantages of CMMI, such as
applying it on an overall level in a multi-disciplinary development.
|
| Q: | Which one is the most difficult of the CMMI levels? |
| A: | If you really start from scratch (from the bottom of level 1), then you
are most likely to find level 2 very difficult. If you manage then to keep
the ball rolling, the other levels will be easier. If you already have a
successful process improvement program based on some other model (e.g. ISO
9000), then you
might be able to move relatively quickly to level 3. Level 4 then is the
next big challenge, as it asks to use statistical methods to control
software development, and thus requires more knowledge on applied statistics
than the average software manager has. Level 4 is a quite tricky level
anyway, because knowing too much about both statistics and the often
creative, unpredictable work of software development makes it difficult to
claim level 4 and keep a clear conscience at the same time. Level 5 is
again relatively straightforward once you are that advanced.
|
| Q: | Why does CMMI come in two flavors: staged vs. continuous representation? |
| A: | There is a religious war in the SPI community on this question, much like Windows vs. Linux. A staged improvement model a la CMM with its levels 1 - 5 gives clear, unambiguous guidance for what to concentrate on next in an SPI program. Critics say, that this is very inflexible, and applying the same approach to a 10 person software house writing embedded software and to a 1000 person organization developing banking software cannot work. They say, the thing to do is to identify the weaknesses of the organization, prioritize them and work on the ones with the highest impact. While this is indeed a good and strong argument, practice has shown that this approach is very difficult to implement for many organizations. There are other points, which are discussed at length and heatedly whenever two or more SPI experts meet. SEI has taken the right decision and provides both versions of its new model, to a large extent with the same contents, and so everyone can choose which one to use. |
Software Quality Assurance (SQA) - FAQ
| Q: | What exactly do you mean by SQA? |
| A: | The term Quality Assurance (and as a matter of fact even Quality)
has a somewhat different meaning in different models. CMMI sees SQA as the verification of process compliance (against an organization or project
standard) and the verification of product compliance (against requirements and
standards). SQA checks if people work the way they are supposed to work,
and SQA checks if what developer produce is really what they are supposed
to produce. This definition shows that SQA brings about more than just
quality in the sense of few defects.
|
| Q: | How does an SQA organization look like? |
| A: | An SQA organization has typically one or more persons trained is SQA
(and able and willing to do it, which is often difficult to find) with at
least some degree of independence from the software developers. They check
process compliance (normally by using checklists) and report the results
to senior management. In case of deviations, senior management, together
with project management and SQA decides what to do. SQA also does some
investigations w.r.t. compliance of the product before it is released to
the customer, and reports its findings. In large organizations, there are
often two or more hierarchical levels of SQA, in accordance with the
structure of the rest of the organization.
|
| Q: | Is quality assurance the process police? |
| A: | SQA is often compared to the police, and it is correct to a certain
degree as it also has this aspect of law enforcement. A good SQA
implementation we would rather compare to a doctor, who makes a diagnosis,
but also helps to cure the problem. And equally important, he would advise
you how to minimize your risks in advance. Often, he is not given the chance
to do so, because he gets involved too late, which is another
common thing with SQA.
|
| Q: | Why do you not consider testing and peer reviews as being quality assurance? |
| A: | These activities are often regarded as the SQA activities of a
project. In CMMI terminology, these activities can be part of SQA, but as
described above SQA is much more. Testing and reviewing is to a large
extent done by people outside the SQA organization, as these are
considered normal development activities.
|
Capability Maturity Model and CMM are registered in the U.S. Patent and Trademark Office. CMMI and CMM Integration are service marks of Carnegie Mellon University.
© Copyright 2001-2009 Q-Success