A thorough explanation of the SE engine and V-process: The V-model derived from decomposition and integration, V&V, and the principles of iteration and recursion.

In the previous article, we explained the fundamental concepts of verification and validation, which form the basis of systems engineering.

Among these, verification and validation are particularly important concepts that form the core of systems engineering.
*From here on, Systems Engineering will be abbreviated as SE.

If you feel that your understanding is unclear, please review the previous lesson's content.

Now, this time we'll finally delve into the specific details of the SE process. When I say "specific details," I don't mean the specifics of individual tasks, but rather an overview of the entire SE process.

Interestingly, the NASA Systems Engineering Handbook, which forms the basis of this article, is itself written from a systems engineering perspective, so it is structured to allow readers to understand the concepts step by step.

The main content of this article is the section on Common Technical Processes and the SE Engine (original title: 2.1 The Common Technical Processes and the SE Engine) from the NASA Systems Engineering Handbook, but in order to understand the SE Engine, it is necessary to understand the concept of V processes.

Therefore, in this article, we will explain the V process (including a review of the previous article), common technology processes, and SE engines in that order.

Let's begin.

What is the V process?

First, let's recall the diagram illustrating the system definition that we explained previously.

As explained two lessons ago, if we consider things as systems based on the definition in the diagram above, we can understand a system as shown in the following diagram.

As concrete examples, I illustrated the automobile, the Japanese government, and pizza delivery services as systems.
*The Japanese government is complex, so we will omit the diagram.

Up to this point, we have reviewed "What is a system? – Understanding the basics of 'systems thinking' through abstraction and MECE," focusing on explaining how to think of existing things, events, and services as systems.

From here, we will consider the development and creation of a system using two specific examples: automobiles and a pizza delivery service.

Develop a system

When developing a new system, such as the cars or pizza delivery services mentioned in the examples, you rarely start by thinking about the screws, springs, and bearings to be used in the new car. Similarly, with pizza delivery services, you probably won't start by thinking about what kind of delivery vehicles to use.

Naturally, you can't decide what to do about components like screws, springs, and bearings until you've determined the specifications of the car itself, including the engine and other components. Similarly, with a pizza delivery service, you can't decide on the details until you've defined the service concept. For example, if the delivery area is a cluster of high-rise apartment buildings, it might be more efficient to use bicycles and delivery personnel instead of vehicles (perhaps outsourcing to Uber Eats).

Thus, unless you first decide on the overall concept—what kind of car to use? What kind of pizza delivery service to offer?—you cannot possibly solve any specific individual problems.

When actually developing a system, people think about things in a step-by-step, concrete way. This process can be illustrated as shown in the diagram below.

In the concrete example of a car, the specifications of the finished vehicle are decided, then the specifications of the powertrain are decided, then the specifications of the engine are decided... and finally the specifications of the individual parts are decided. In the case of a pizza delivery service, the service overview is decided, then the details of the delivery are decided, and then the specific method of transportation is decided, and so on, in that order.

This is where it becomes very interesting for system engineers: the different stages of system development correspond to the different levels of abstraction when things are broken down into systems.

This can be translated into terms commonly used in development as follows:

This is a key concept in SE (Systems Engineering), demonstrating a step-by-step development approach. Of course, this isn't done in a single subsystem or sub-subsystem; development proceeds simultaneously and in parallel across each subsystem and sub-subsystem.

Simply put, it's a diagram illustrating what people do unconsciously. Conceptually, it represents what we want to achieve with the system.It is broken down step by step, separating it into numerous individual components.Therefore, this becomes a process of increasing the number of elements.

Create a system

Now let's think about actually building the system.

Here, we'll consider a system where development is complete, and the final components are finished. Naturally, screws, springs, and bearings alone cannot make a complete car. Similarly, in a pizza delivery service, the entire service cannot be made up of just the delivery vehicles.

By assembling individual components and combining delivery vehicles, personnel, and equipment according to a specific purpose, a subsystem (sub-subsystem) is created that fulfills a particular objective.

These completed components (subsystems, sub-subsystems) are gradually integrated with other components (subsystems, sub-subsystems) to create things like automobiles and pizza delivery services.

When this step-by-step assembly and combination is generalized using the concept of a system, it looks like the diagram below.

This step-by-step assembly and integration process is similar to the "system development process" in that each stage corresponds to a corresponding level of abstraction when things are broken down into a system.

Let's rewrite this in more general terms, similar to "develop a system."

Interestingly, the process is the exact opposite of "developing a system," and is fragmented.Integrate the peripheral elements step by stepWe will proceed. What's interesting about this creation process is that, conceptually, it's the exact opposite of the process of "developing a system." We combine and integrate numerous peripheral elements step by step, so ultimately it converges into a single system.

V Process

Up until now, we've considered developing a system and building a system as separate processes.

Since we're at it, let's combine these two processes. However, simply combining them would make it difficult to read, so we'll arrange them in a V shape.

This allows the entire process of "developing and creating a system" to be represented in a single diagram.

Now, let's slightly modify the diagram by replacing the words "implementation" and "integration" in the "system creation" process on the right side with "testing."
*In order to actually implement and integrate the system, it is necessary to verify it through testing before proceeding to higher-level processes.

This is what you commonly see as the V process.

The V process can be described simply as follows:

It may seem like common sense, but when you're actually working as part of a team to implement a surprisingly complex system, it's easy to lose sight of "what you're doing and why," so this is a very important concept.

V-process, Verification, and Validation

Since we have this opportunity, let's use the V process to enhance the image by performing important verification and validation checks on the content from last time.

First, let's review my key points.

Let's start by considering the verification process.

Verification, by definition, is the process of checking how things actually are in relation to the goal. In the context of the V process, this is equivalent to checking whether the test results on the right side meet the specifications set in the design on the left side.

Therefore, it can be illustrated as follows:

In the V process, does verification involve checking whether the left and right sides are consistent?This is very easy to understand and clearly shows when and what to check.

However, whether or not the overall vertex test is for verification will be determined later.

Next, we will consider validation.

The definition of validation is: How does the objective relate to the goal? Therefore, if we consider it in terms of the V process...Whether the higher-level processes and lower-level processes are logically and correctly connected.?This will be confirmed.

To give a concrete example using the V process, the top left shows whether the objectives of the basic design align with the goals derived from the requirements definition. The top right shows whether the functions of the constituent elements (modules) are properly fulfilling their roles within the overall system.

This can be illustrated as follows:

In the V process, validation is very easy to understand as it represents the consistency between the upper and lower hierarchical levels. Similar to verification, the diagram makes it clear "what to check and when" during validation.
*However, validation is difficult because it is not a task that can be judged quantitatively using numerical values ​​or other means.

The remaining issue here is the connection between the requirements definitions on the left and right sides of the vertex and the overall test. Based on the theory so far, it seems like verification would be the logical approach, but considering the meaning...This is a check to see if the requirements for the entire system match the system that has been created, thus it is a validity check.This is what happens.

In other words, since a system that has been created cannot be confirmed to be completely completed as expected through mere testing, it is necessary to verify it by using the system in a real environment, which is why it is called validation.
*Please remember that even after thorough final testing, some minor issues may still occur in the market.

This can be illustrated as follows:

You can see that even with left-right consistency, only the vertex can be used for validity verification.

The verification and validation processes described so far can be represented as a single V process as follows:

This verification and validation process is included in the V process when building a system.When, at what point, and what should be checked?This will make it easier to understand.

This diagram greatly supports the "art" of systems engineering, which we discussed last time: "knowing when and where to investigate" in order to pursue system balance and verify its validity.

このThe verification and validation processes are referred to using the first letters of each other as V&V.Sometimes called.

My personal impression is that in Japanese companies, projects are generally broken down into individual tasks and managed along a timeline (often using Gantt charts).

While this method is ideal for checking the progress and achievement of each individual task, it makes it very difficult to understand how the progress and achievement of each task impact the overall project. Especially often overlooked is the coordination and verification between tasks. Currently, this coordination and verification skill largely relies on individual experience and skills.

Here are some real-life examples of V&V failures in Japan.

There are countless other problems as well.
*The system design is flawed, so no matter how hard each person in charge tries, the problem is difficult to solve.

As a solution, alongside Gantt charts, breaking down a project and managing it using a V-process makes the connections between each task visible, which helps reduce variations in individual progress, verification tasks, omissions, and skill differences, greatly improving work efficiency. I highly recommend it.

It's effective to use them in a way that you can manage the progress of individual tasks using a Gantt chart and the progress of the entire project using the V-process.
*If you would like to know more about practical methods, detailed techniques, or ideas, please contact me directly.

I've gone off on a tangent, let's get back on track.

Understanding the Importance of Interfaces in the V Process

Here, we will use the V process to gain a deeper understanding of the importance of managing the connections between interactions between elements—that is, managing interfaces and flows—which was one of the key topics covered in the previous lesson.

To review, let's recall the specific examples of disassembling an automotive system and flow interfaces.

Let's consider what happens when this flow and interface management is missing, using the V process.

If we proceed vertically without considering the flow and interface at all, we can proceed without any problems up to the point shown in the following diagram in the V process (vertical work).

If you don't implement flow interface management, this is where the real trouble begins.

Now, this is the worst-case scenario. The units, modules, and systems cannot be assembled or will not function as expected, so we will have to start all over again.

When you illustrate it like this, it might seem obvious, but I'm sure many of you have had similar experiences at work or elsewhere, right?

Even in my own engine design experience, mismatches in interfaces or flow paths are often fatal and require a complete rework. It's rare for these issues to be resolved by simply modifying parts (such as mismatched mounting dimensions or load transmission paths).

Here are a few famous examples of major interface management failures in Japan.

There are so many examples that it would be impossible to list them all.
* Some problems persist because the system design is flawed, making it difficult to resolve them despite the best efforts of each team member.

This clearly demonstrates the crucial importance of managing flows and interfaces when developing or creating systems.

The actual methods of flow interface management will be explained in detail in later articles and chapters of this SE series.
*If you would like to know how to do it immediately, please contact me directly.

From what we've discussed so far, it should be clear that the role of the systems engineer, as explained last time, plays a very significant role in the success of a system (in this explanation, we'll focus on the importance of V&V and interface management).
*I believe that the root cause of most of Japan's specific failures lies in the lack of people who fulfill a role equivalent to that of a systems engineer.

This concludes the section on the V process and the content covered in the previous lesson, aiming to deepen your understanding.

Having grasped the overall system description, we will now explain what is actually done at each level, primarily using the translation of the Common Technical Processes and the SE Engine section (original title: 2.1 The Common Technical Processes and the SE Engine) from the NASA Systems Engineering Handbook.

Common technology processes and SE engines

NASA's NPR 7123.1 (NASA Systems Engineering Processes and Requirements) classifies common technical processes into three sets: "system design, product realization, and technical control."
*NPR 7123.1 is similar to internal business regulations or internal technical standards in a typical Japanese company.

The content of each of these processes, their interactions, and the flow of information are illustrated in NPR.System Engineering EngineThe SE engine process is used to develop and realize the final product (a complete system such as a service or organization).

*Details of the diagram's content will be explained in later chapters and articles, so a general understanding is sufficient for now.

Processes 1 through 9 shown in the diagram represent specific "tasks (practical work)" in project execution, while processes 10 through 17 serve as "cross-cutting tools" for carrying out those practical work.

System Design Process

The four processes in the system design process are used for the following purposes:

These processes are applied to each level of the system, from the highest level to the lowest.

The lowest-level elements of the system are defined until they can be "built, bought, or reused."
* "Manufacturable, purchasable, or reusable" refers to a level at which something can be concretely materialized. For example, for a product, it would be at the component level; for an organization or service, it would be at the specific level of individual people, tasks, or rooms.

The lowest layer (the entity level) is created through implementation, and the layers above that are realized by integrating those entities to create the system.

Product Realization Process

The product realization process (processes 5-9) is applied to products within the system, starting from the lowest level of entities and progressing towards the higher-level integrated products.

This product realization process is used to create design solutions for each product (purchasing, coding, manufacturing, or reusing existing materials), verify and validate the product to meet the design solution and stakeholder expectations, and then move it to the next level of hierarchy.

Technical management process (cross-cutting process)

The technical management processes (processes 10-17) are used for the following purposes:

The system design process, product realization process, and technical management process within the SE engine are:Iterative and recursiveIt is used repeatedly.

Definition of iterative and recursive

NPR 7123.1 defines iterative and recursive as follows:
*NPR 7123.1 is like an internal company standard, similar to what you'd find in a Japanese company.

This also applies when repeatedly applying the same process to the system in the next lifecycle phase to mature the system definition and meet the success criteria for each phase.

The technical process is applied recursively and iteratively to break down the initial concept of the system into a level of detail specific enough for the technical team to implement the product.

Subsequently, the smallest product is integrated into the larger system, and the process is again applied recursively and iteratively until the entire system is assembled, verified, validated, and migrated (system completion).

Compatibility with AS9100 (Compatibility with standards for the civil aerospace industry)

AS9100 is a quality management system standard developed and widely adopted for the civil aerospace industry.
*This is a standard that all companies involved in the aerospace industry must comply with. Compliance is required even at the lowest component level.

Some NASA centers have chosen to obtain AS9100 certification and may require their contractors to comply with NPR 7123.1.

For more detailed examples of how to actually use the SE engine (Systems Engineering Engine), please refer to the separate document, "NASA Expanded Guidance for SE".
*Specific examples of the extended version will be introduced in a separate article on this site.

Explanation of Common Technology Processes, SE Engine, and V Process

Here, let's deepen our understanding by linking the common technology processes and SE engine explained in Chapter 2 with the V process.

Similar to the explanation in Chapter 2 of this article, we will divide the process into system design process, technical management process, and product realization process, and then combine them with the V process.

System design process and V process

First, we apply the system design process to the V process.

As can be seen from the diagram, the system to be designed becomes increasingly concrete and subdivided by carrying out the system design process at each level. The requirements, technical requirements, theoretical definitions, and design solution definitions that have been concretized in the higher-level system design process are passed down to the lower-level system design process and proceed from there.

Ultimately, these become multiple component drawings and detailed specifications, which, as described in Chapter 2 of this article, reach a level where they can be manufactured, purchased, or reused, and are then actually procured.

Specific examples of manufacturing, purchasing, or reusable components in the automotive industry include individual parts such as bolts, springs, and bearings, which are actually manufactured, procured, or purchased. However, what's interesting here is that when an automotive company procures components it doesn't manufacture itself, such as an ECU, there's no need to break down the elements beyond the ECU level, so the system design process is completed at the ECU level (functionally purchased components).

Thinking about it in terms of a pizza delivery service, if you outsource the delivery of the pizzas to a company like Uber Eats, you don't have to worry about the delivery vehicles or delivery personnel; it's the same logic (you're giving Uber Eats a specification document for what you want them to do).

Technical management processes and V processes

Next, let's consider the technical management process.

The technical management process acts as a command center.

The technical management process is always in conjunction with each system design process, managing and controlling each system design process (requirements, technical risks, configuration, technical data management and evaluation).

This technical management process optimizes the overall balance by ensuring that the design of each subsystem and sub-subsystem is properly controlled (validation and interface management).

For example, if a person or department with strong authority unilaterally decides on specifications and proceeds with things based on their own strong interpretation, this technical management process will keep them in check (they must intervene). Otherwise, the project will deviate significantly from the initial plan, and in the worst case, it will fail and be canceled.
*Many people have probably experienced a situation where a department unilaterally changed the interface, making it impossible to set up the system.

Conversely, for designs handled by departments lacking resources, budgets, or personnel, the technical management process provides proactive support to maintain overall balance.

In other words, This technical management process is extremely important, as it involves looking at the big picture—the core of a systems engineer's role—and balancing it to optimize the system.It will be exactly that.

Based on my personal experience and observations, I believe the biggest reason why the introduction of SE and V-process concepts in Japan has not yielded the desired results is that the technical management process is insufficient and those responsible for managing the technical management process lack sufficient authority.

Product Realization Process and V Process

Next, we apply the product realization process to the V process.

In the product realization process, just like in system design, the same product realization is carried out at each stage according to the level of the design. What is interesting here is that, unlike the system design process, numerous minimal components are progressively integrated into a single system.

The final completed system is considered complete after going through the requirements definition and validation steps in the upper left of the V process during the product realization process.
*The SE process itself is not finished, as there is a lifecycle that will be explained next time.

Here, I'll also include the product realization process and the common technology management process in the same diagram.

The technical management process is always linked to each product realization process and manages and controls each product realization process (e.g., verification, management of validation results).

However, the role of the technical processes shown in the diagram above is only during normal operation; their true value is revealed when problems occur in each product realization process, making them crucial processes to play a vital role.

For example, as shown in the diagram above, if a problem occurs during testing of a component (unit) in the product realization process, the technical management process analyzes the nature of the problem, organizes and incorporates new insights, formulates a new technical plan, and restarts the process from the appropriate level.
*If the requirements are completely unfeasible, we may need to revise the requirements definition from the highest level.

This clearly demonstrates his role as the team's strategist.

Common technology processes, SE engine and V process

The diagram below summarizes the relationship between each process and the V process.

The diagram has become quite complex,Where, how, and what each process produces.I think this will be easier to understand. In particular, you will see that the technical management process complements the role of managing all processes in all directions, both vertically and horizontally.

As explained in Chapter 2 of this articleThe technical management process is cross-cutting.I think this makes it easier to understand.

Explanation of iterative and recursive V-processes

Finally, since the concepts of iteration and recursion are very important, we will use the V process to further understand them.

Iterative and V-process

Let's start by considering the concept of iteration. Let's reiterate the definition of iteration.

Since the trigger condition for iterative actions is "discrepancies or deviations from requirements found," this is equivalent to a problem occurring somewhere, as explained in the V process above.

When that problem occurs, you will be instructed to run the process again to fix it.

Based on this, we will illustrate the repetition with a diagram.

As the diagram shows, for example, if a problem occurs during testing of a component (unit), you shouldn't just fix it on the spot and move on, but instead repeat the process as needed.
* Determining the appropriate stages, quantity, and scope of the process is where the technical management process truly shines, and where systems engineers can demonstrate their skills.

This is the meaning of iteration in SE (Systems Engineering). Of course, the diagram uses a problem during the testing phase of a component (unit) as an example, but the process is repeated regardless of when the problem occurs.

The main reason for iterative work in SE is to prevent the overall balance from being disrupted, as making corrections only at specific levels or parts would be equivalent to performing localized optimization.

In my experience, localized fixes rarely have a significant impact on the whole system, leading to much bigger problems. I'm sure you've all experienced situations with games and other software where no matter how many patches are applied, the problem remains.

By handling even localized problems across the entire system, it's possible to maintain optimal balance.

Recursion and V-process

Next, let's consider recursion.

First, let's write out the definition of recursion again.

Since "recursive" is not a word we use very often, I will explain its meaning in Japanese.

The dictionary definition of "recursive" in Japanese seems to be "the state in which something refers to or includes itself again." Since that's a bit difficult to understand, how about rephrasing it in my own words as "the state in which one looks back at oneself and changes"?

From a personal perspective, the key point is that "you yourself change."

Based on this, we will illustrate recursion in SE. The scene we will illustrate is the left side of the V process, specifically the case where validation is performed during the design process.

As you can see from the diagram, it's not simply a matter of going back up the hierarchy, but rather going back up the hierarchy and changing oneself, in other words, it's recursion.
* Determining the appropriate stages, quantity, and scope of recursion is where the technical management process truly shines, and it's a chance for systems engineers to demonstrate their skills.

In system engineering, a major purpose of recursion is to prevent the entire system from failing due to a stack at a specific level or location.
*To put it in extreme terms, this prevents the entire project from being forced to shut down due to the failure of the design of a single component.

This recursive approach is an important concept for resolving localized problems by involving higher levels and resolving them across the entire system.

Iterative and recursive V-process

Up to this point, we have explained iterative and recursive methods independently. Here, we will combine them and explain the important concepts of iterative and recursive methods in software engineering.

Let's get straight to the diagram. The diagram will illustrate a scenario where a problem occurs during testing of a component (unit).
*This is the same as the pattern within the product realization process and V process described in Chapter 3 of this article.

As can be seen from the diagram, if a problem occurs during testing of a component (unit), instead of proceeding with a corrective action, the process is iterated, and each level is recursively advanced.
* Determining the appropriate stages, quantity, and scope of iteration and recursion is where the technical management process truly shines, and where systems engineers can demonstrate their skills.

This is the meaning of the important concept in SE: "executing a process iteratively and recursively."

To reiterate, the key point here is that the main mechanism aims for overall optimization, rather than just local optimization of the trouble hierarchy (or specific part), by iteratively and recursively re-executing the process, rather than simply making corrections.

This is one of the major factors that ensures overall optimization, which is the greatest characteristic of SE. Personally,A self-evolving process where the entire system is optimized by learning from problems within the process.I think so.

In my personal opinion, many Japanese companies and organizations are good at iterative work but quite bad at recursive work (especially at higher levels).

The mechanism is that Japan's "once a decision is made, it must be seen through" mentality, combined with the considerable authority of those in charge at higher levels, amplifies the negative effects of these two factors, making recurrence at the higher levels extremely difficult.

Here are a few representative examples in Japan that lack both iteration and recursion.

There are far too many other issues to list, and some are ongoing problems.

Perhaps the most significant impact on Japan was the complete power outage at the Fukushima Daiichi Nuclear Power Plant (since the case of a complete power outage hasn't been recursively tested, there's no established response).

I believe that this representative problem from the past clearly demonstrates the importance of repeating iterative and recursive processes, and I sincerely hope you will understand this.

Summary

Although this is supposed to be a summary, this article is quite lengthy and contains important information, so I'd like to briefly review the content and conclude by offering some advice based on my own experience.

First up is the V process.

The V process became popular in the early 2010s and is now a common concept, but its essence is rarely explained, so I have provided a thorough explanation here.

V. A practical tip for the process is not to use the conceptual diagram as is, but to actuallyIt is important to write down your company's and your own tasks and apply them to the V-process.I think it 's.
*The first step in anything is to write it down and then take action.

By actually writing out the V process in detail, I believe we can implement effective V&V and interface management that is relevant to real-world situations.

These days, the V process is sometimes said to be outdated and unable to keep up with speed, but my feeling is that it's just imitating the V form without doing the crucial V&V and interface management.

New business models keep emerging one after another, but from what I've seenThe basics are the V process and SE concepts.Therefore, I believe that understanding and practicing V-processes and SEs will always be important, regardless of the era.

Next are the V process and the common technology process/SE engine.

The most important aspect of the V-process and the common technology process/SE engine is the cross-cutting technology management process.

If this technical management process doesn't function properly, introducing system engineers (SEs) often won't be very effective. It's important to note that if the central command center isn't functioning correctly, no matter how much effort is put into each system design and product realization process, the overall effect will be largely meaningless.

I believe that the introduction and establishment of the technical management process, which serves as the command center, will be a major key to success.

Finally, let's talk about iterative and recursive processes.

The most important aspect of this article is the iterative and recursive execution of the process.

This forms one of the core elements of the SE process. I've explained many details up to this point, but I think even just introducing this iterative and recursive process alone will yield significant results.

Even if the contents of the common technology process and SE engine cannot be followed exactly, simply iterating and recursively through the process will often be quite effective.

Personally, if you want to maximize the effect with minimal changes, then the conventionalThe practical processes were reorganized by role, including SE verification and validation, interface management, and iterative and recursive processes.establishmentThat seems like a reasonable compromise.
*The key to success for implementers and consultants lies in integrating traditional business processes with SE processes. Please feel free to contact me if you have any questions.

That concludes our in-depth explanation of the SE engine and V-process—the V-model derived from decomposition and integration, V&V, and the principles of iteration and recursion.

Next time, I'll explain the lifecycle.

• NASA Systems Engineering Handbook Rev2
  This book is the basis for this explanation. It is in the public domain, so you can get it for free.

• Systems Engineering Handbook, Keio University Press, supervised by Hidekazu Nishimura
  We'll be working hard to supervise the NASA handbook on this site, but it will take time, so if you want to learn the whole thing right away, I recommend this book. I own a copy myself.