# System Engineering. <br/> Architectural Modeling of Computer Systems ## Lecture 6-2 ### Quality in Engineering Aleksandr Penskoi, 2025 ---- ### Lecture plan 1. Quality as Conformance to Requirements 1. Computer Systems Differences 1. Quality is Related 1. Economics of Quality 1. Quality Management Model 1. Culture Patterns --- ## Quality as <br/> Conformance to Requirements In traditional engineering: **"Quality is free."** That means: - A system should be in strict accordance with its technical requirements. - If it is (and we can check it by measurements) <br/> — it will be a high-quality system. ---- ### Quality is Free <br/> Based on the Following Principles 1. The definition of quality is conformance to requirements <br/> (requirements meaning the product and the customer's requirements) 1. The system of quality is prevention 1. The performance standard is zero defects (relative to requirements) 1. The measurement of quality is the price of nonconformance --- ## Computer Systems Are Different We can't fully describe requirements at the early stages. Main reasons: 1. We have a very short production cycle <br/> but some systems work and are maintained for more than half a century. 1. Development is a multidisciplinary project. 1. We have a huge design space with an infinite number of options. 1. Incredibly complex toolchains and modules (OS, frameworks, etc.). 1. We have an unprecedented opportunity for our systems' distribution <br/> e.g. mobile phone distribution  <!-- .element height="200px" --> --- ## Quality is Relative Quality is **the value to some person**. The system quality is a tradeoff: - more quality for one person - may mean less quality for another - (which can change during the system's evolution). ---- ### Quality is Relative (other points of view) In Essence terms: quality is the level of satisfaction with the system by **significant stakeholders**. In a funny form, the system quality is described by **the principle of least surprise** (the system should cause the least possible shock to a stakeholder). ---- ### Quality Simplification - Working with the computer system's quality should be a dynamic consensus between many controversial requirements. - Simplification of quality understanding to a simple model can harm the system. ---- #### Examples of Bad Quality Simplifications 1. Zero defects are high quality, e.g.: - a user of our text processor can work with a bibliography, but if it grows more than a thousand the system will crash. 1. Having lots of features is high quality, e.g.: - Emacs, Microsoft Word, and distraction-free editor. 1. Elegant coding is high quality, e.g.: - developers can polish code but why should a business pay? 1. High performance is high quality, e.g.: - improve performance by rewriting from Python to C, it requires months and increases maintenance costs. 1. Low development cost is high quality 1. Rapid development is high quality 1. Safety and security is high quality 1. User-friendliness is high quality 1. Reliable system is high quality --- ## Economics of Quality - Usually, an **error** in computer systems is **determined** at a **late stage** (integration, acceptance testing). - At the same time, it was **committed at the early stages** (requirements analysis, architectural design). - The **more delay** we have between making an error and the moment of its detection, the **higher the cost** of correcting it will be. - In the limit: it may cause a complete system redesign. ![Dependency between error cost and life cycle stage []](./figures/error-cost-from-stage.png) <!-- .element height="300px" --> ---- ### How to Manage Quality Price? <div class="row"><div class="col"> 1. That puts us in front of the development methodology tradeoff between: - design (working with requirements and architecture) - and development (writing source code and hardware specifications). 1. In corner cases we can: - design our system (until it is unnecessary to everyone) - or completely redesign our system after each meeting with customers (and, of course, never end). </div><div class="col">  What is the Best Solution? It depends on... </div></div> --- ## Quality Management Model 1. The software quality determines the **entire development process**: - from selecting development tools, modules, and design methodology - to coding and quality control. 1. Methods and tools of quality control, including verification methods, must be incorporated into a development process. 1. Quality control of the output result only is not adequate in most cases. > Poor management can increase software costs more rapidly than any other factor. > <br/> — (c) Barry W. Boehm ---- ### Management in Control System Style <div class="row"><div class="col">  </div><div class="col"> - **Controller** — project management that should observe the development process and fix it if something goes wrong. They can **change** project **resources** or system **requirements**. - **Development system** — development team that **produces** the computer system and also changes their competencies, morale, culture, etc. They consume resources and random factors, like dependency issues, market changes, or illness. </div></div> ---- ### Management System <div class="row"><div class="col">  </div><div class="col"> - Need to know: - What kind of product is wanted? - By what processes can such a product be made? - What's actually being made, and by what process? - And two conditions: - There must be visible evidence of how the process is going on. - The process must be stable enough so that the evidence is meaningful. </div></div> --- ## Culture Patterns in Organizations **Things are the way they are because they got that way.** - We can identify at least six culture patterns for computer system development companies: Oblivious, Variable, Routine, Steering, Anticipating, and Congruent (the last two usually are theoretical). - It is not a maturity model. - Any software pattern can be a success for the right project/customer.  ---- ### Pattern 0. Oblivious *We don't even know that we're performing a process.* - Pattern 0 is not a professional pattern, but we add it because: - it is the most frequent source of new programs, - and it can be used as a baseline against which other patterns can be compared. - There is no software development organization separate from the software user. - Whether because of fun, cognitive dissonance, or some other factor, Pattern 0 is highly successful at producing satisfied users. ---- ### Pattern 1. Variable *We do whatever we feel like at the moment.* 1. This pattern could be called the pattern of the Individual Programmer. 1. There is no knowledge of management as a development tool. 1. Usually, this pattern is present in young companies producing software products for microcomputers or in a large organization where a pool of programmers serves some important group of specialists. 1. Pattern 1 organizations may have some procedures, but they don't cover most parts of the actual process. 1. Besides, they always abandon any procedures at the first sign of crisis. 1. The best predictor of project schedule, cost, or quality is the specific programmer who does the job. The programmer gets all the credit, as well as all the blame. ---- ### Pattern 2. Routine *We follow our routines (except when we panic).* 1. For a big project or big company, where we can't achieve results with a small team. In any case, managers decide they can no longer afford to leave the programmers alone. 1. "Beginning to recognize that quality management can help but is unwilling to devote time and money to make it happen." There are several reasons they don't provide the money or time: - They don't appreciate the value of what can be accomplished. - They don't know what is needed to accomplish changes. - They believe that pushing developers is all they need to do the job. 1. A programmer said of his managers, "They think they're managing a salami factory." 1. The ideal development structure is a manager supported by powerful tools and procedures. When the jobs are routine, all the manager has to do is ensure that everyone does every step in the right order. ---- ### Pattern 3. Steering *We choose among our routines by the results they produce.* 1. Pattern 3 managers never depend on magic but on understanding. Although there are many exceptions, the average Pattern 3 manager is more skilled or experienced than the average Pattern 2 manager. 1. They have a variety of skills required to steer an organization, so they don't have to fall back on yelling when their project gets in trouble. 1. Their procedures are not always completely defined, but they are always understood. 1. Managers generally follow the processes they have defined, even in a crisis. That's why they can successfully manage larger, riskier projects with a greater degree of success. 1. If you examine the "typical" project, Pattern 3 may not look spectacularly better than Patterns 1 and 2. In Pattern 3, however, more projects are "typical" because there are far fewer outright failures. ---- ### Pattern 4. Anticipating *We establish routines based on our past experience with them.* 1. A Pattern 4 manager is similar to the Pattern 3 manager but sits at a higher level in the organization and has a higher level of understanding concerning quality management. 1. Instead of merely reacting to instabilities, the organization anticipates them and acts in advance. 1. Pattern 4 managers have procedures that they understand and follow uniformly. 1. The organization has initiated comprehensive process measurements and analysis. This is when the most significant quality improvements in individual projects begin. ---- ### Pattern 5. Congruent (not found in the real world) *Everyone is involved in improving everything all the time.* 1. At Pattern 5, quality management moves to the highest level or optimal performance. 1. Managers consider high-quality management an essential part of the company system. 1. Level 5 organizations will understand and follow procedures, which everyone is involved in improving all the time. 1. This provides the organization with a foundation for continuous improvement and optimization of its process.