Skip to main content

By Mr M. Latta

 

The society that separates scholars from its warriors will have its thinking done by cowards and its fighting done by fools.” Thucydides

 

Thinking is mental activity that allows us to understand, plan, reason, solve problems, innovate and make decisions[1]. It is concerned with how we take in, process and utilise information. In the military, it forms a large part of the conceptual component of fighting power[2]. Military leaders (and their staff) are often faced by unbounded, complex, ill-defined problems with high time pressure and high stakes for failure[3]. They must adopt thinking styles and methods that allow them to make appropriate plans and decisions to solve the problems they are faced with.

The situation is further complicated in that military leaders may at various times make complex plans by themselves or in a group, with an extended timeframe or with very little time to consider, about problems they are familiar with or ones that they know very little about, when information is available or when it is scant[4]. The military professional must be able to recognise how the situation may affect thinking requirements, and to adapt thinking methods as required.

In 2013, Tactical School commenced a review of the staff and tactics training delivered at the school to commissioned officers in the NZ Army. The review started ‘from zero’, examining the job requirements of officers at Captain and Major levels (in terms of command staff demands and all-arms tactical knowledge)[5]. As part of this, the statement that ‘we train people how to think, not what to think’ was examined, resulting in the question being asked: “How do we want people to think?” Considerable effort was put into researching and analysing the thinking process in the military, and how best to train it in officers, resulting in a substantial body of information being developed[6]. Of particular relevance to this article, was the identification of an emphasis across many militaries and corporates on two ‘modes’ of thinking, critical and creative.

The modes of thinking

The modes of thinking describe the purpose behind the thinking. Critical thinking aims to ensure the robustness of information to aid a decision; it is a process of intellectual rigour[7]. Creative thinking aims to generate new ideas to aid a decision; it is a process of active imagination[8].  At Tactical School, on the Grade 3 Staff and Tactics course the focus is on developing critical thinking skills, while creative thinking skills are developed on the Grade 2 Staff and Tactics course.

Critical thinking is a mode of thinking in which critical processes are applied to a situation to determine implicit elements as well as the explicit, examine the elements, and their soundness. The structure of information is examined, and the logic of the structure, so as to form judgements that provide guidance on further actions[9]. In a military sense, critical thought is essential to ensure decisions are made on objective information and not emotion or unfounded intuition.

Creative thinking is a mode of thinking in which creative processes are applied to a situation to generate ideas to provide novel and original options for further action. This can be through examining a situation through a new or alternate perspective to break preconceptions, adaption of existing aspects to unintended purposes through innovative insight, or development of new ideas through invention and experimentation. In a military sense, creativity enables a commander to achieve surprise and adapt faster to new conditions than an adversary.

The modes are not mutually exclusive: it is a case of critical thinking and creative thinking, not critical thinking versus creative thinking. “[Creative thinking] masters a process of making or producing, [critical thinking] a process of assessing or judging. The very definition of the word ‘creative’ implies a critical component…when engaged in high quality thought, the mind must simultaneously produce and assess, both generate and judge the produce it fabricates. In short, sound thinking requires both imagination and intellectual standards.”[10] Critical thinking also requires imagination to determine what factors may not have been considered.

Critical Thinking[11]

Critical thinking aims to ensure the robustness of information to aid a decision; it is a process of intellectual rigour. One of the primary functions of command is to make plans, decisions, or otherwise solve problems that eventuate. These may be caused by the natural environment, threat elements, or hazards posed by military operations. Critical thinking helps to ensure that those decisions or plans are based on the best use of the best information.

In adopting a critical thinking mode, thinking is a search for clarity, precision, and accuracy in information; where all aspects of an issue are considered in a non-biased manner; so that the conclusions reached are well reasoned, logical, and based on evidence. Critical thinking involves analysing and evaluating one’s own thinking and that of others. It examines a problem in depth from multiple points of view, and involves determining whether adequate justification exists to accept conclusions as true based on a given inference or argument. Overall, it is an analytical process of ensuring sound logical reasoning lies behind any information applied to a situation.

Specific behaviours associated with critical thinking include exploring issues and ideas to determine vital questions to answer, both implicit and explicit, and ensuring there is not a bias in the information being presented or used. Defining statements and questions clearly and precisely to provide adequate focus to aid understanding; and gathering and analysing relevant information using abstract ideas to interpret it effectively are also critical actions. Additionally, evaluating the value and weight of the information, based on an informed judgement of the evidence relating to it; reaching well-reasoned conclusions and solutions, and testing them against relevant criteria and standards; and openly considering alternative systems of thought, are behaviours consistent with critical thinking.

Logic

Logic is a key concept in critical thinking. Logic is defined as reasoning conducted or assessed according to strict principles of validity and soundness[12]. Logical reasoning breaks an argument or line of reasoning down into component parts, known as propositions.   Propositions are simply statements that can be true or false, however in examining logic they will be either a premise or conclusion. A premise is a statement that provides support to an argument’s conclusion. A conclusion is the statement that is the final decision of the argument or line of reasoning.

Logical reasoning is valid if its conclusion logically follows from its premises. Otherwise, it is said to be invalid. Note that the premises don’t necessarily have to be true to have valid reasoning. If they are true, and the reasoning is valid, then the reasoning is also sound. If either of those conditions does not hold, then the argument is unsound. Truth of a premise or conclusion is determined by looking at whether they are in accordance with facts in the real world.

A logic chain is a link from a premise or premises to a conclusion. A conclusion may then become a premise in a subsequent line of argument, creating a chain of logic.   Put another way, each premise may be made up of a number of sub-premises. A chain is only as strong as its weakest link, and therefore if any premise is untrue in reality, then the conclusion reached is unsound. This is a flawed logic chain. Similarly, if the conclusion reached does not actually follow on from the premise (even if the premise is true) then the reasoning is invalid, and is also flawed logic.

Logical reasoning can take one of three forms based on the relationship of the premises to the conclusion. The three forms are: deductive reasoning, inductive reasoning, and abductive reasoning. Deductive reasoning is reasoning from one or more premises to reach a logically certain conclusion. If correct deductive reasoning is used, if all premises are true the conclusion must be true. Inductive reasoning is reasoning from one or more premises to reach a logically probable conclusion. The conclusion does not follow with any necessity from the premises, meaning it is theoretically possible that the conclusion is untrue, but it is probable given the premises. Abductive reasoning infers a set of premises as a result of a conclusion. It is therefore acutely prone to fallacy (untruth) because there could be a number of premises.

It can be identified that deductive reasoning is actually relatively rare in military planning given the difficulty in finding out the truth or in getting complete information about the type of complex problems dealt with. Most often inductive or abductive logic is being used instead, sometimes being referred to (wrongly) as deductive. A process of logical reasoning is essential to the military professional to ensure thinking arrives at sound conclusions. A disciplined approach will also identify where risks exist in the thinking, given either the form of thinking that is used or the presence of assumptions in formulating the conclusion. In commonwealth armies, the format most often applied to the discipline of logical thought is the Three Column Analysis or Three Column Format.

Application at Tactical School – the Grade 3 course

On the Grade 3 Staff and Tactics course (Lieutenants transitioning to Captain), the focus is on ensuring sound logic in thinking. The approach taken is that there is no right or wrong answer in military tactics, only answers that are more likely to have sound thought applied to them. Student’s tactical solutions are examined on eight criteria[13] and the student questioned as to their logic in each area. Doing this ensures that the tactical concepts and actions being planned are sound; one cannot have a sound plan without sound logic.

An example of the application of logic to tactics is the use of doctrinal principles. Doctrine is a guide, not a set of rules, and should be judiciously applied as required. Over time, templated solutions to specific problems have been developed that fulfil doctrinal principles relevant in that particular case. There can be a tendency for the application of those templates to become considered as the application of doctrinal principles. However, in some similar problems, elements of those doctrinal principles, or perhaps the way they are applied in the template, are not relevant. The blind application of a template to a problem is a constraint on precise thinking, so Tactical School now requires students to explain their logic in applying doctrinal principles and the actions that fulfil them. They are allowed not to apply doctrine in cases where there is sound logic behind that choice. This ensures there is critical thought and sound logic behind how students consciously apply doctrine to tactical problems.

The requirement for sound logic extends to the conduct of planning in a staff environment. At Tactical School in the staff planning exercises, the emphasis is on effective information management and student’s ability to share relevant information across other staff branches is one aspect that is assessed. This relevant information may be conclusions that have been made in the individual student’s own area of planning (for example potential actions by the enemy for students in the intelligence cell) which are required premises for other areas of planning (for example actions within the plan by students in the operations cell). In effect, the students, operating as a staff headquarters, are required to support a logic chain that spans the entirety of the staff branches.

By establishing and making explicit the requirement for sound logic in deliberate planning activities, students are better enabled to conduct intuitive problem solving[14]. The application of intuition to problem solving involves making leaps of logic, essentially moving multiple steps along a logic chain to reach a conclusion faster than if required to work through all the premises sequentially. This is not without risk, but in certain cases the time gained in making decisions faster is worth that risk[15]. In order to reduce the risk, the intuitive process (or system 1 processes) should be based on experience relevant to the situation[16]. By working through decisions at Tactical School, and the logic involved in them, students develop the ability to make intuitive leaps as their experience grows[17].

Creative Thinking[18]

Creative thinking aims to generate new ideas to create or aid a decision; it is a process of active imagination. Military forces consistently try to get an advantage over an opposing element. Being able to do unexpected, new, and innovative actions enables a military force to achieve an advantage over the adversary. This may be a technical innovation, or a novel tactical course of action, or anything that achieves an element of surprise over the adversary to which they are unable to react effectively. Creative thinking is also the foundation to adapting to (and solving the problem of) the adversary’s innovation.

In adopting a creative thinking mode, thinking is a search for new ideas or approaches; where judgement is suspended to explore possibilities; so that many potential options are generated without being constrained by existing paradigms. Creative thinking involves un-restraining one’s own thinking and that of others. It explores possibilities within a problem, and linking or associating apparently disparate ideas, in a process of continual experimentation and imagination. Overall, it is an exploratory process of developing multiple possibilities, including those aspects original in entirety, combination, or application.

Specific behaviours associated with creative thinking include an ability to accept change and newness, while not accepting that the status quo is the best or only way, therefore pursuing continual improvement. Additionally, associating ideas or elements across a diversity of fields of thought or skill, developing novel and original insight into issues and problems, and pursuing randomness or deliberately breaking preconceptions and accepted norms are also creative actions. Exploring the limits of standard judgements and norms, a willingness to experiment with ideas and possibilities, and acceptance of failures and mistakes as part of the process are also behaviours associated with creative thinking. Finally, creativity also includes elaborating on ideas and concepts to build detail, depth, originality, and add value; and evaluating the suitability of ideas and designs to determine the appropriateness and usefulness of them.

Unbounded Thinking

Many of the techniques of creative thinking are about allowing unrestrained, free thought that can explore outside of the boundaries of norms, conventions, or standard thinking. It is often self-imposed barriers that inhibit creativity, and therefore a key concept to enable creative thinking is to push beyond those barriers. By looking at issues from alternative perspectives, combining apparently unrelated ideas, or refusing to accept the status quo or common explanations, often new concepts are created.

Many of the psychological mechanisms that occur naturally in our thinking reinforce compliance with norms and inhibit variance, meaning that the mind itself is geared to reduce the ability to be creative in many ways. These mechanisms are known as heuristics and biases[19]. Heuristics are mental rules of thumb that allow us to make the rapid mental calculations that are necessary for quick decisions and responses. They are short-cuts, tried and tested in survival situations, but as they lack rigor they do not work in all situations. They act as a sort of mental anchor that makes it difficult to escape their influence.  Biases are intended to create efficiency in thought by pre-disposing us to give more weight to information that leads us in a desired direction; however they can be a risk if they lead us in the wrong direction. We are unaware of many of our biases which contain both innate and learned aspects.

It can take a lot of discipline to overcome those heuristics and biases and exercise unbounded thinking.  Specific techniques are taught at Tactical School to assist in this, and the concept of incubation is also taught which refers to a temporary break from problem solving that can result in insight, i.e. ‘sleeping on it’. This break seems to allow the unconscious means of thinking to work, which can assist in breaking down boundaries of thinking. This can be difficult to justify in a time-constrained environment, such as there often is in the military, however it is important to allow sufficient breaks in conscious thinking and accept that thinking is still occurring at an unconscious level. If ideas are not forthcoming, it is likely to be more productive to stop trying to ‘force’ them and allow some time for them to incubate.

While many heuristics can be a risk in thinking, one heuristic that is useful to creative thinking is the ‘Naïve Diversification heuristic’. Due to this heuristic, if asked to make several choices at once we tend to create broader and more distinct options than if making the same type of decisions sequentially. Thus, three courses of action developed simultaneously by a commander (or staff) to answer an operational problem will be more diverse than three developed sequentially[20]. A modification made to the Military Appreciation Process as applied at Tactical School attempts to harness the power of this heuristic, and is discussed later in this article.

The specific creative thinking techniques used at Tactical School to assist in overcoming heuristics and biases affecting creative thinking include out-of-box thinking, provocation, and reframing. Out-of-box thinking is a technique that forces alternatives to be generated, in which a ‘box’ is placed around the concept, idea, or piece of information for which alternatives need to be generated. All idea generation after that must assume that the information inside the box is to be avoided or unavailable – this can be particularly useful in a military context where a different, less obvious solution is being sought as it often is in tactics. The provocation technique involves developing a solution that is so unrealistic that it is ridiculous. This technique is used to ‘break through’ any barriers to thought and allow radical solutions to be identified which may then be modified to workable solutions. ‘Reframing’ refers to looking at a problem from a different perspective, or a different ‘frame’ of view. In Reversal Reframing, that viewpoint is an opposite one. Alternative Perspectives Reframing adopts multiple perspectives or foci in looking at the problem.

These techniques can be used to generate a broad range of option from which to then develop critically robust solutions. Emphasis is on quantity rather than quality at this stage, any judgement about the value of the options generated should be suspended until it is decided to start re-imposing those boundaries. The discipline required to exercise creative thinking is no less than that which is required for critical thinking. Indeed, in many ways more discipline is required because society at large, and military culture in particular, tends to emphasise critical thinking and de-emphasises behaviour that supports creativity and innovation.

Application at Tactical School – the Grade 2 Course

The Grade 2 Staff and Tactics course is a requirement for Captains transitioning to Major in the New Zealand Army. With the premise that the Grade 3 course will have established a foundation of critical thinking (and logic), the Grade 2 looks to develop creativity in military solutions. Specific creative techniques are taught, and then the students guided through their application in the Military Appreciation Process.

Early emphasis on supportive and non-judgemental class work aims to promote an environment where creativity is not stifled or shut down. As students on the Grade 2 are assessed on leading a staff branch, this aspect of leadership behaviour within a command staff environment is an important factor in developing officers that will facilitate novel and adaptive solutions in the staff environment. In terms of assessing creativity, analysis of training determined that the eight criteria used to assess tactical plans was still valid and could be used to support creativity. For example, a novel approach was more likely to achieve surprise or seize the initiative, a key factor in the ‘applies the Manoeuvrist Approach’ criterion.

This is not to suggest that ‘wild’ ideas are sought, as any design student knows creativity must acknowledge the laws of the real world, and so too must the tactical solutions produced on the Grade 2. That is why the foundation of critical thinking is so important. The creative and unbounded solutions produced at the early stages are gradually honed using the judicious application of logical reasoning to develop a solution that is both novel and realistic. Anecdotally though (as only one new model course has run) the variety of solutions developed by students is greater than on previous courses, particularly in the staff planning exercises where the application of the creative process results in more distinct courses of action to select from than before.

Critical & Creative Thinking and the Military Appreciation Process  

The Military Appreciation Process (MAP) is the doctrinal process that the Army follows when conducting planning. It varies from individual application to group application[21], and from Army-centric application to Joint application[22]. The core elements in doctrine (in individual application[23]) consist of six steps: Mission Analysis, Battlespace Analysis, Threat Analysis, Course of Action (COA) Development, COA Analysis, and lastly Decision and Execution. In order to facilitate the application of critical and creative thought, Tactical School teaches the process as nine steps, in four stages.

The first stage, framing, sees the problem ‘framed’ and the scope of required analysis determined. It involves one step – receipt of mission. The receipt of mission step frames the problem to focus on the key elements of it. In this step critical thinking is employed to define the problem, check information regarding own force readiness, and analyse timelines. Creative thinking is used by recognising that an unbounded free-thought occurs as the individual is made aware of the situation, and this may result in valuable initial thoughts to aid subsequent analysis which should be captured. This step should also, in both a critical and creative sense, see factors (premises) identified for further evaluation in the analysing stage.

This step is a new addition to the MAP in New Zealand, although similar steps exist in United States[24] and British[25] Armies planning models. It developed from a review of those processes and as a result of analysing what effective problem solvers did anyway. The emphasis on staff planning that was recently revised training at Tactical School has also required the addition of this step. This was done to formalise a requirement for commanders and chiefs of staff to provide appropriate guidance and focus to their staff when initiating planning.

The second stage, analysing, involves three steps – mission analysis, battlespace analysis, and threat analysis[26]. The analysis stage is particularly heavy on critical thinking, drawing conclusions relevant to the formation of a plan. Mission Analysis is the process of determining what must be done, including evaluating the available guidance on how it should be done, in order to meet the commander’s requirements. Battlespace Analysis is the examination of the physical and non-physical environment in which the mission is to occur. Threat Analysis[27] examines the adversary or adversaries in the mission to determine what adversary actions are possible, given their capabilities and the terrain, and identifies vulnerabilities that may be targeted to defeat them. The logical reasoning applied to analysing the different factors in the mission, battlespace, or threat is fundamentally critical thinking. However, identifying how those factors create opportunities is a creative aspect, as is identifying all the potential relationships between factors.

The third stage is forming, where multiple viable options are actually formulated, and involves both creative and critical thinking in action. It consists of four steps – course of action (COA) design, COA test, COA development, and COA analysis. COA design sees multiple COA concepts developed that can achieve the mission and defeat the threat, which are then compared against adversary actions in COA Test. This determines possible reactions and decision by both sides and confirms the viability of the COA for further development in the COA Development step. A fully developed COA is then analysed against the adversary in COA Analysis, culminating with the evaluation and comparison of options for the final step of Decision and Execute.

The COA Design and COA Test steps are Tactical School modifications from the original process designed to stimulate creative thinking and inculcate it formally within the planning process. In COA design, multiple different broad concepts are devised simultaneously, creating a point in the appreciation process where the naïve diversification bias is intentionally put into play. The use of the creative thinking techniques outlined earlier are encouraged, to identify all the possible courses of action that could be undertaken without unconsciously dismissing possibilities because of self-imposed judgements. As a formal step, it facilitates the removal of boundaries to aid creative thought by creating ‘safety’ in recognising that reality will be re-imposed later in this and subsequent steps.

COA Test is a modification which sees an earlier combination of friendly and adversary actions than in the standard appreciation process. It was introduced to prevent a recurring problem identified in planning of developing friendly and adversary plans without adequately catering for how specific actions by the opposition would shape those plans.  In essence, plans were developed fully in isolation only to be found invalid at the COA Analysis step. The COA Test step then is a critical thinking imposition to ensure that the conclusions (actions in the plan) are based on sound premises (responses by the opposition).

COA development sees the validated COA concept fleshed out to incorporate all aspects, in an equally creative and critical manner. This is then further assessed in the COA Analysis step. The final stage is confirming, with a single step – decision and execution. This sees a COA selected for implementation and a move from the ‘plan’ stage of the operations cycle[28] to the ‘prepare’ and ‘execute’ stages.

Conclusion

Thinking is fundamental to what Officers and military leaders do. Leader’s thinking will either keep soldiers alive or get them killed, so professional military leaders should be as good at thinking as they can possibly be. Military leaders (and their staff) must adopt thinking styles and methods that allow them to make appropriate plans and decisions to solve the problems they are faced with.   Critical thinking is a process of intellectual rigour that aims to ensure the robustness of information to aid a decision; for the military it is essential to ensure decisions are made on objective information and not emotion or unfounded intuition. Creative thinking is a process of active imagination that aims to generate new ideas to aid a decision; for the military creativity enables a commander to achieve surprise and adapt faster to new conditions than an adversary.

Tactical School training develops a foundation of critical thinking and logical reasoning in students on the Grade 3 Staff and Tactics Course. This is followed by fostering creative, unbounded thinking on the Grade 2 Staff and Tactics Course. The application of cognitive science to current proceedings has seen the Military Appreciation Process modified to aid, amongst other things, the application of both critical and creative thought in NZ Army officers.

Endnotes

[1] JDN 3/11, Decision-Making and Problem solving: Human and Organisational Factors, London, England: Ministry of Defence, 2011.

[2] NZDDP-D, New Zealand Defence Doctrine (3rd ed.), Wellington: New Zealand Defence Force, 2012.

[3] J. Pounds, and J.J. Fallesen, Problem solving of mid-career Army Officers: Identification of general and specific strategies. ARI Research Note 97-21. Fort Leavenworth: United States Army Research Institute for the Behavioural and Social Sciences, 1997

[4] Ibid.

[5] For more details refer to the position description developed, available on the Tactical School intranet site (NZDF internal only).

[6] Tactical School, Tactical School Guide: Thinking (ver 2), NZ Army, 2016.

[7] All explanations regarding Critical Thinking are directly from Tactical School Guide: Thinking (Ver 2); however are a composite from a number of sources as follows:

ADRP 6-0, Mission Command, Washington, DC: Department of the Army, 2012.

Edward M. Glasser, An Experiment in the Development of Critical Thinking, Columbia: Teacher’s College, Columbia University, 1941.

  1. Paul and L. Elder, The Miniature Guide to Critical Thinking: Concepts and Tools, The Foundation for Critical Thinking, 2009

Red Teaming Guide (2nd ed), London, England: Ministry of Defence, 2013.

  1. Scriven and R. Paul, Critical Thinking as Defined by the National Council for Excellence in Critical Thinking, presented at the 8th Annual International Conference on Critical Thinking and Education Reform, 1987.

The Applied Critical Thinking Handbook (Version 7), Fort Leavenworth: University of Foreign Military and Cultural Studies, 2015

TH!NK program, www.think.dasa.ncsu.edu,  Raleigh, NC: North Carolina State University, accessed July 2015.

[8] All explanations regarding Creative Thinking are directly from Tactical School Guide: Thinking (Ver 2); however are a composite from a number of sources as follows:

ADRP 6-0, Mission Command, Washington, DC: Department of the Army, 2012.

Edward De Bono, Serious creativity: Using the power of lateral thinking to create new ideas. HarperBusiness, 1992.

Robert Harris, www.virtualsalt.com, accessed July 2015

  1. Koestler, The Act of Creation, London: MacMillan, 1964.

Jonathan Milne, Go! The Art of Change, Wellington: Steele Roberts Aotearoa Ltd, 2008.

Red Teaming Guide (2nd ed), London, England: Ministry of Defence, 2013.

TH!NK program, www.think.dasa.ncsu.edu,  Raleigh, NC: North Carolina State University, accessed July 2015.

[9] Paul and Elder.

[10] Paul and Elder

[11] Refer note 7

[12] Oxford English Dictionary

[13] The criteria are: utilises a Manoeuvrist approach, produces a sustainable plan, complies with relevant doctrine, achieves the requirements of the higher commanders, defeats the adversary’s assessed actions, maximises the effects of the available capabilities, adheres to the fundamentals of command, consciously accepts and manages risk.

[14] Gary Klein, Sources of Power: How people make decisions, Cambridge, MA: MIT Press, 1999.

[15] R. Flin and K. Arbuthnot, Incident command: Tales from the hot seat. Aldershot, UK: Ashgate, 2002.

[16] Klein.

[17] Part of the rapidity of the German forces invading at the commencement of WW2 is attributed by some historians to them having practiced planning activities (TEWTs or staff rides) such as river crossing that they then faced in actual operations.

[18] Refer to note 8

[19] JDN 3/11 and J. Beshears and F. Gino, Leaders as decision architects,  in Harvard Business Review, 93(5), pp52-62, 2015.

[20] JDN 3/11

[21] LWD 5-1-4, The Military Appreciation Process, Canberra: Australian Army, 2009.

[22] ADFP 5.0.1, Joint Military Appreciation Process, Canberra: Australian Defence Force, 2015.

[23] LWD 5-1-4

[24] ADRP 5-0, The Operations Process, Washington, DC: Department of the Army, 2012.

[25] AC71038, The Staff Officers’ Handbook 2014, Warminster: British Army, 2014.

[26] Battlespace Analysis and Threat Analysis steps are the first and second halves, respectively, of the Intelligence Preparation of the Battlespace (IPB) process as outlined in LWD 5-1-4.

[27] If non-threat groups are also analysed, this step may be termed stakeholder analysis as stated in LWD 5-1-4.

[28] Plan, Prepare, Execute, Assess (ADRP 5-0)