source: Nancy Leveson, Engineering a Safer World: Systems Thinking Applied to Safety Why practise complex technologies hence oftentime...
.PNG)
source: Nancy Leveson, Engineering a Safer World: Systems Thinking Applied to Safety
Why practise complex technologies hence oftentimes fail, too neglect inwards such unexpected ways? Why is it hence hard for hospitals, chemic plants, too railroads to pattern their processes inwards such a agency equally to dramatically cut down the accident rate? How should nosotros sweat to furnish systematic analysis of the risks that a given engineering presents too the causes of accidents that sometimes ensue? Earlier posts accept looked at the ways that sociologists accept examined this occupation (link, link, link); but how practise gifted engineers address the issue?
Nancy Leveson's electrical flow book, Engineering a Safer World: Systems Thinking Applied to Safety (2012), is an outstanding introduction to organisation security engineering. This mass brings frontward the pioneering piece of work that she did inwards Safeware: System Safety too Computers (1994) amongst novel examples too novel contributions to the champaign of security engineering.
Leveson's basic insight, hither too inwards her before work, is that technical failure is rarely the termination of the failure of a unmarried component. Instead, failures termination from multiple incidents involving the components, too unintended interactions amidst the components. So security is a characteristic of the organisation equally a whole, non of the private sub-systems too components. Here is how she puts the signal inwards Engineering a Safer World:
Safety is a organisation property, non a element property, too must hold upwards controlled at the organisation level, non the element level. (kl 263)Traditional peril too failure analysis focuses on specific pathways that Pb to accidents, identifying potential points of failure too the singular "causes" of the accident (most usually including operator error). Leveson believes that this approach is no longer helpful. Instead she argues for what she calls a "new accident model" -- a ameliorate too to a greater extent than comprehensive agency of analyzing the possibilities of accident scenarios too the causes of actual accidents. This novel excogitation has several of import parts (kl 877-903):
- expand accident analysis past times forcing consideration of factors other than element failures too human errors
- provide a to a greater extent than scientific agency to model accidents that produces a ameliorate too less subjective agreement of why the accident occurred
- include organisation pattern errors too dysfunctional organisation interactions
- allow for too encourage novel types of peril analyses too peril assessments
- shift the emphasis inwards the role of humans inwards accidents from errors ... to focus on the mechanisms too factors that shape human behavior
- encourage a shift inwards the emphasis inwards accident analysis from "cause" ... to agreement accidents inwards damage of reasons, that is, why the events too errors occurred
- allow for too encourage multiple viewpoints too multiple interpretations when appropriate
- assist inwards defining operational metrics too analyzing performance data
So thinking well-nigh accidents inwards damage of component failure is a serious misreading of the nature of the technologies amongst which nosotros interact every day. Instead she argues that security applied scientific discipline must hold upwards systems engineering:
The solution, I believe, lies inwards creating approaches to security based on modern systems thinking too systems theory. (kl 88)One of import constituent of a ameliorate agreement of accidents too security is a recognition of the fact of complexity inwards contemporary engineering systems -- interactive complexity, dynamic complexity, decompositional complexity, too nonlinear complexity (kl 139). Each of these forms of complexity makes it to a greater extent than hard to anticipate possible accidents, too to a greater extent than hard to assign discrete accident pathways to the occurrence of an accident.
Accidents are complex processes involving the entire sociotechnical system. Traditional event-chain models cannot pull this procedure adequately. (kl 496)Leveson is highly critical of iterative security applied scientific discipline -- what she calls the "fly-fix-fly" approach. Given the severity of outcomes that are possible when it comes to command systems for nuclear weapons, the operations of nuclear reactors, or the air traffic command system, nosotros require to hold upwards able to practise ameliorate than exactly improving security processes next an accident (kl 148).
The model that she favors is called STAMP (Systems-Theoretic Accident Model too Processes; kl 1059). This model replaces the linear component-by-component analysis of technical devices amongst a system-level representation of their functioning. The STAMP approach begins amongst an sweat to seat crucial security constraints for a given system. (For example, inwards the Union Carbide institute at Bhopal, "never allow MIC to come upwards inwards contact amongst water"; in pattern of the Mars Polar Lander, "don't allow the spacecraft to deport upon the planet surface amongst to a greater extent than than a maximum force" (kl 1074); inwards pattern of populace H2O systems, "water lineament must non hold upwards compromises" (kl 1205).) Once the constraints are specified, the number of command arises; what are the internal too external processes that ensure that the constraints are continuously satisfied? This devolves into a laid of questions well-nigh organisation pattern too organisation administration; the instrumentation that is developed to stair out compliance amongst the constraint too the direction systems that are inwards house to ensure continuous compliance.
STPA has ii primary steps:
- Identify the potential for inadequate command of the organisation that could Pb to a hazardous state.
- Determine how each potentially hazardous command activeness identified inwards stair 1 could occur. (kl 2758)
.PNG)
It would hold upwards really interesting to come across how an engineer would employ the STAMP too STPA methodologies to evaluate the risks too hazards associated amongst swarms of autonomous vehicles. Each vehicle is a organisation that tin hold upwards analyzed using the STAMP methodology. But likewise the workings of an freeway amongst hundreds of autonomous vehicles (perhaps interspersed amongst less predictable human drivers) is equally good a organisation amongst complex characteristics.
Each private vehicle has a hierarchical organisation of command designed to ensure prophylactic shipping of its passengers too the vehicle itself; what are the failure modes for this command system? And what well-nigh the swarm -- given that each vehicle is responsive to the other vehicles or hence it, how volition private cars response to odd circumstances (a jack-knifed truck blocking all 3 lanes, let's say)? It would appear that autonomous vehicles create the kinds of novel hazards amongst which Leveson begins her mass -- complexity, non-linear relationships, emergent properties of the whole that are unexpected given the expected operations of the components. The fly-fix-fly approach would propose the deployment of a sure enough number of experimental vehicles too hence evaluate their interactions inwards real-world settings. Influenza A virus subtype H5N1 to a greater extent than disciplined approach using the methodologies of STAMP too STPA would brand systematic efforts to seat too command the pathways through which accidents tin occur.
Here is a fake swarm of autonomous vehicles:
But accidents happen; neither software nor command systems are perfect. So what would hold upwards the termination of 1 disabling fender-bender inwards the intersection, followed past times a one-half dozen more; followed past times a gigantic pileup of robo-cars?
COMMENTS