Abstract
We present a large class of security protocol abstractions with the aim of improving the scope and efficiency of verification tools. We propose abstractions that transform a term’s structure based on its type as well as abstractions that remove atomic messages, variables, and redundant terms. Our theory improves on previous work by supporting rewrite theories with the finite-variant property, user-defined types, and untyped variables to cover type flaw attacks. We prove soundness results for an expressive property language that includes secrecy and authentication. Applying our abstractions to realistic IETF protocol models, we achieve dramatic speedups and extend the scope of several modern security protocol analyzers.
Introduction
Security protocols play a central role in today’s networked applications. Past experience has amply shown that informal arguments justifying the security of such protocols are insufficient. This makes security protocols prime candidates for formal verification. In the last two decades, research in formal security protocol verification has made enormous progress, which is reflected in many state-of-the-art tools including AVANTSSAR [5], ProVerif [9], Maude-NPA [21], Scyther [14], and Tamarin [34]. These tools can verify small to medium-sized protocols in a few seconds or less, sometimes for an unbounded number of sessions. Despite this success, they can still be challenged when verifying real-world protocols such as those defined in standards and deployed on the internet (e.g., TLS, IKE, and ISO/IEC 9798). Such protocols typically have messages with numerous fields, support many alternatives (e.g., cryptographic setups), and may be composed from more basic protocols (e.g., IKEv2-EAP).
Abstraction [10] is a standard technique to over-approximate complex systems by simpler ones to make verification more efficient or feasible. Sound abstractions preserve counterexamples (or attacks in security terms) from concrete to abstracted systems. In the context of security protocols, abstractions are extensively used. Here, we only mention a few examples. First, the Dolev–Yao model [19] is a standard (but not necessarily sound) abstraction of cryptography. Second, many tools encode the verification problem in the formalism of an efficient solver or reasoner. These encodings often involve abstraction as well. Therefore, we call these back-end abstractions. For example, ProVerif [9] translates models in the applied pi calculus to a set of Horn clauses, SATMC [6] reduces protocol verification to SAT solving, and Paulson [40] models protocols as inductively defined trace sets. Finally, some abstractions aim at speeding up automated analysis by simplifying protocols within a given protocol model before feeding them to verifiers [28,37]. Our work belongs to this class of front-end abstractions.
Extending Hui and Lowe’s work [28], we proposed in [37] a rich class of protocol abstractions and proved its soundness for a wide range of security properties. We used a type system to uniformly transform all terms of a given type (e.g., a pattern in a protocol role and its instances during execution) whereas [28] only covers ground terms. Our work [37] exhibits several limitations: (1) the theory is limited to the free algebra over a fixed signature; (2) all variables have strict (possibly structured) types, hence we cannot precisely model ticket forwarding or Diffie–Hellman exchanges. While the type system enables fine-grained control over abstractions (e.g., by discerning different nonces), it may eliminate realistic attacks such as type flaw attacks; (3) some soundness conditions involving quantifiers are hard to check in practice; and (4) it only presents experimental results for a single tool (SATMC) using abstractions that are crafted manually.
In this work, we address all the limitations above. First, we work with rewrite theories with the finite-variant property modulo a set of axioms to model cryptographic operations. Second, we support untyped variables, user-defined types, and subtyping. User-defined types enable the grouping of similar atomic types (e.g., session keys) and adjusting the granularity of matching in message abstraction. Furthermore, we have separated the removal of variables, atomic messages, and redundancies, from the transformation of the message structure. This separation simplifies the specifications and soundness proof of the abstractions that transform the message structure. Third, we provide effectively checkable syntactic criteria for the conditions of the soundness theorems. Finally, we extended Scyther [14] with fully automated support for our abstraction methodology. The resulting tool is available online [36]. We validated our approach on an extensive set of realistic case studies drawn from the IKEv1, IKEv2, ISO/IEC 9798, and PANA-AKA standard proposals. Our abstractions result in very substantial performance gains. We have also obtained positive results for several other state-of-the-art verifiers (ProVerif, CL-Atse, OFMC, and SATMC) with manually produced abstractions.
This article is based on the conference paper [38] from which it differs mainly as follows. On the theoretical side, we have generalized the class of supported rewrite systems from a subclass of shallow subterm-convergent ones to all those with the finite-variant property. Using the finite-variant property, we have significantly simplified the condition needed for equality preservation (Theorem 4.23). On the practical side, we provide additional details of the abstraction heuristics and the implementation. We have also extended the Scyther implementation with a check for spurious attacks. Moreover, we have performed several additional case studies.
Due to space constraints, most proofs are moved to the full version of the paper [39]. Table 1 gives an overview of the rest of the paper.
Structure of paper
Structure of paper
The Internet Key Exchange (IKE) family of protocols is part of the IPsec protocol suite for securing Internet Protocol (IP) communication. IKE establishes a shared key, which is later used for securing IP packets, realizes mutual authentication, and offers identity protection as an option. Its first version (IKEv1) dates back to 1998 [27]. The second version (IKEv2) [30] significantly simplifies the first one. However, the protocols in this family are still complex and contain a large number of fields.
Concrete protocol. As our running example, we present a member of the IKEv2 family, called IKEv2-mac (or
We consider the following security properties:
the secrecy of the DH key mutual non-injective agreement on the nonces
The DH key serves as the master secret for
Abstraction. Our theory supports the construction of abstract models by removing inessential fields and operations using a range of abstractions. Typically, we use abstractions in a first step to remove selected cryptographic operations, remove fields under hashes, and to pull fields outside other cryptographic operations like encryptions or signatures. The types enable a fine-grained selection of the messages to be abstracted. In a second step, we remove inessential top-level (i.e., unprotected) fields and redundancies.
Let us apply these two steps to the
In a second step, we use abstractions to remove the fields o, A, B,
Scyther verifies the properties (P1) and (P2) in 8.7 s on the concrete and in 1.7 s on an automatically generated abstract protocol (which differs somewhat from the one presented here). Our soundness results imply that the original protocol
Security protocol model
We define a term algebra
The set of message terms is
Type system
We introduce a type system akin to [2] and extend it with subtyping. This type system is very fine-grained. For example, there are different types for different fresh values. We will subsequently restrict some abstractions to apply only to arguments of a specific type. Thus, the purpose of this fine-grained type system is to control when those abstractions are used. The subtyping allows us to adapt to different setups and tools by making types more coarse-grained. For example, we can define a type
We define the set of atomic types by
We assume that all variables have an atomic type, i.e.,
The subtyping relation
Every type is a subtype of
Equational theories
An equation over a signature Σ is an unordered pair
A rewrite rule is an oriented pair
Provided that
A rewrite theory
The equality
We model the protocols of our case studies (see Section 2 and Section 6.2) in the rewrite theory
The theory of XOR is given by the following rewrite system. The rightmost rule is redundant but required to ensure coherence [29].
We have used the AProVE termination tool [23] and Maude’s Church-Rosser and coherence checker [20] to verify the termination, confluence, and coherence properties that are required for decomposing the equational theories of our case studies.
Finally, we define well-typed substitutions, which are substitutions that respect subtyping.
A substitution θ is well-typed if
Since the type of any variable is atomic, this definition is independent of the representative of the
The finite variant property
The finite variant property simplifies equality checking and unification in equational theories. Given an equational theory Consider the XOR theory from Example 3.3 and the terms For our theoretical development, we consider an arbitrary but fixed equational theory
We also call
We specify a security protocol as a partial function from agent variables to roles. A role is a sequence of events. We distinguish three types of events: send events, receive events, and signal events. A send event
Given a set of terms T, we define the set of events A protocol is a partial function We formalize the (Protocol).
(
Operational semantics
In this section, we introduce an operational semantics for security protocols. This semantics specifies the dynamic behaviour of the protocol roles when their events are executed. The protocol messages are sent to and received from the adversary, whom we identify with the network as usual.

Intruder deduction rules (where
We use a Dolev–Yao adversary model parametrized by an equational theory E. Its judgements are of the form
When a protocol is executed, each of its roles can be executed an arbitrary number of times by possibly different agents in parallel. Such a single execution of a role is called a thread. We distinguish between different threads by associating each thread with a unique thread identifier. We index variables and fresh values with the thread identifier i to syntactically distinguish them from those of other threads. This ensures the uniqueness of fresh values.
Let
For example, suppose that thread i plays role A and is owned by
We thus define the set of network messages exchanged during protocol executions by
Given a protocol P, we define a transition system with states
The trace

Operational semantics.
The rules in Fig. 2 define the transitions. The first premise of each rule respectively states that a send, receive, or signal event heads thread i’s role. This event is removed and added together with the thread identifier i to the trace
Finally, we define the semantics of a protocol P with respect to the intruder’s initial knowledge
We assume that the intruder’s initial knowledge
This assumption specifies the minimal requirements. The attacker usually also knows the long-term shared and private keys of the compromised agents and the public keys of all agents, i.e., the keys in
(Example trace).
We provide an example trace of a partial honest execution. In this trace, Alice performs a partial session with Bob, up to the point of Bob’s
In this trace, the adversary does not interfere. There are also traces in which he does interfere, e.g., traces in which the adversary sends the first message. In such traces, the first event could be a responder receive, for a suitable choice of σ in the initial state.
Property language
Meier et al. [33] define a predicate-based security property language. In this language, many security properties such as those from [13,16,32] can be specified. In this section, we introduce a specification language for security properties based on [33]. Our language is similar to the languages used in [1,21,26].
Syntax. Our property specification language is an instance of first-order logic with formulas in negation normal form (i.e., only atomic formulas can be negated). Let
The atomic predicates and their informal meaning are as follows, where
Semantics. We define the semantics of our language
Let
For properties ϕ, we write
In the following example, we present our formalizations of secrecy and authentication properties for the We express the secrecy of the Diffie–Hellman key We formalize non-injective agreement of A with B [32] on the nonces Note that our property language does not allow expressing the general notion of injective agreements as defined by Lowe [32], which amounts to counting the numbers of An alternative formulation of our protocol semantics and property language, suggested by one of the reviewers, is obtained by viewing each variable and fresh value as an unary function symbol and keeping the thread identifier variables as the only variables of the property language. The set of network messages would thus become The substitutions We see two possibilities for dealing with protocol specifications in such an approach. The first possibility is to keep protocol specifications unchanged, i.e., using messages from The second possibility is to also use messages from In both cases, the operational semantics and numerous details would have to be carefully adapted. We believe that our setup strikes a good balance between an economic notation for protocol specifications and a uniform treatment of different kinds of messages in our abstraction theory. (Properties of
In this section, we present two kinds of protocol abstractions:
transform a term’s structure by removing or reordering fields and by removing or splitting cryptographic operations. The types enable a fine-grained selection of the transformation to apply. The same transformation is applied to all terms of a given type and its subtypes. complement typed ones with two additional kinds of simplifications: atom/variable removal abstractions and redundancy removal abstractions. The former remove unprotected atoms or variables while the latter remove terms that the intruder can derive.
In Section 4.1, we give an overview of the different kinds of abstractions and their combined use. We then proceed with the formal definitions and results for our protocol abstractions that we will apply in the following chapters. Our main results are soundness theorems for the typed and untyped abstractions. They ensure that any attack on a given property of the original protocol translates to an attack on the abstracted protocol. Similar to [28], we follow a modular approach for proving this property. We first define a general notion of protocol abstraction for which we prove a general soundness theorem under certain conditions (Section 4.2). These conditions concern the preservation of intruder deducibility as well as of equalities and disequalities. We then go on to define each concrete kind of abstraction and prove its soundness (Sections 4.3–4.5). We illustrate the usefulness of our definitions on our running example. For the soundness proofs it then suffices to establish the conditions of the general soundness theorem. As we will see, each such soundness result in turn imposes certain conditions, which we will introduce and motivate by examples.
Upon first reading, readers may choose to skip the remainder of this section after reading the following overview and proceed to the next sections to get an impression of how we will use the abstractions.
Overview
Typed abstractions are our main mechanism to simplify the cryptographic structure of terms by removing protections that are not required to achieve a given property. We specify typed abstractions by a list of recursive equations. The following example illustrates a range of typical forms of defining equations. Messages are transformed according to the first matching pattern. If no pattern matches then the top-level symbol is transformed homomorphically. Typed abstractions leave atoms and variables untouched.
(Typed abstractions).
Consider a simplified variant of the
Let us now apply this typed abstraction to the fourth message in A’s role. We elide the application of f to atoms and variables (where f is the identity) and on pairs,
Generally speaking, in order to preserve the deducibility of messages, typed abstractions cannot remove fields that are extractable (e.g., by projection, decryption, or signature verification), whereas removing the non-extractable fields under a hash-type function such as
(Atom-and-variable removal).
Applying the typed abstraction above to the fourth protocol message yielded
(Redundancy removal).
Since
We have chosen to factor out the removal of atoms and variables as well as redundancies from the typed abstractions, since this substantially simplifies their definition and soundness proofs.
General soundness theorem for protocol abstractions
We start by defining a general form of protocol abstraction that encompasses all of our concrete abstractions. We then prove a general soundness theorem for these abstractions, which we later instantiate to obtain concrete soundness results.
General protocol abstractions
A general protocol abstraction consists of two functions. The first functions transforms the terms in the protocol definition and in protocol executions, while the second one transforms properties. For some but not all concrete abstractions these functions will coincide. We introduce the set
(General protocol abstraction).
A (general) protocol abstraction is a pair For events: For event sequences: For the atomic predicates of our property language:
Although general protocol abstractions have two independent fields, our concrete typed and untyped abstractions will use only special forms. For typed abstractions and atom-variable removal abstraction, we will have
Soundness of general protocol abstractions
To justify the soundness of our abstractions
We will define the substitution
Let
(Safe formulas).
Let g is the identity function on for all for all for all
Condition (a) ensures that
We now state the soundness theorem for the general abstractions.
(General soundness theorem).
Let P be a protocol, ϕ a property,
For all states
For all states
ϕ is safe for P and
Then for all states
Condition (i) ensures that derivability is preserved for received messages. Similarly, condition (ii) ensures the deducibility preservation for claimed secrets. Condition (i) is needed to establish conclusion 1 and conditions (ii) and (iii) are required for conclusion 2. Below we sketch the proof of this theorem. The full proof can be found in the full version [39].
To show point 1 (reachability preservation), let
To show point 2 (attack preservation), we proceed by induction on the structure of ϕ and use assumptions (ii) and (iii). □
In the following subsections, we discuss each kind of protocol abstraction and the associated soundness result. For these proofs, it suffices to define the function g and to establish the conditions (i)–(iii) of the theorem above. In each case we introduce the assumptions that are needed for this purpose and motivate them by examples.
Our typed abstractions are specified by a list of recursive equations subject to some conditions on their shape. We define their semantics in terms of a simple Haskell-style functional program. We use both pattern matching on terms and subtyping on types to select the equation to be applied to a given term. This ensures that terms of related types are transformed in a uniform manner.
Syntax and semantics
Let
A function specification
For
The function specification
We use vectors (lists) of terms
A typed abstraction is a function specification of the form
The concatenation of
We present a typed abstraction

Program f resulting from
The semantics of a typed abstraction the term t matches the pattern p, i.e., its type
The first enabled clause is executed. Hence, the equations
We will often identify the typed abstraction
Consider the typed abstraction given by the first three equations from Example 4.10, including the types of the variables. Suppose we would like to use the associated program f (as specified in Program 1) to abstract the term
Finding abstractions is fully automated by our tool using a heuristic that we will describe in Section 5. To show a concrete application of typed abstractions to our running example while giving a first idea of our heuristics, we use here the following simplified abstraction strategy: We start by identifying the terms that appear in the To preserve the secrecy of the DH key (from
Soundness of typed abstractions
We now turn to showing the soundness of the typed abstractions. We do this by establishing conditions (i)–(iii) of our general soundness theorem (Theorem 4.6). The main ingredients that we need for this purpose are the preservation of intruder deduction, equalities, and disequalities. These properties will not hold without restrictions on the protocol, the property, and the typed abstraction. We first formulate these properties, their scope, and introduce these restrictions informally. We then state our soundness theorem. We defer the detailed motivation and formal definitions of the restrictions to the subsequent subsections.
For the correct interpretation of the properties of typed abstractions, it is important to remark that, given a term
Suppose σ is the substitution component of a concrete state, Deducibility preservation. Here, we require that
Equality preservation. This means that
Disequality preservation. This can be formulated as the reverse direction of equality preservation:
To establish these properties, we will use the following substitution property, which we will discuss in detail in Section 4.3.4. For terms t and well-typed and
Finally, we can state our soundness result for typed abstractions.
Let
Then for all states
It suffices to establish conditions (i)–(iii) of Theorem 4.6 for
Let
To prove that condition (iii) of Theorem 4.6 is satisfied, we have to establish conditions (a)–(e) in Definition 4.5. We look at each of these conditions in turn.
Condition (a): holds trivially since Condition (b): clearly holds since σ is well-typed and f is the identity on atoms. Condition (c): Here, Condition (d): holds by assumption (iii). Condition (e): holds by assumption (iv).
This completes the proof of the theorem. □
In the following, we discuss each of the properties (P1)–(P4) in more detail. We give examples motivating the restrictions under which they hold and we formally define these restrictions. We then establish that the properties hold under the respective restrictions. We start our discussion with the substitution property. Readers who wish to first get an overview of our abstractions before delving into the technical details may want to skip to Section 4.4.
The following example shows that the substitution property does not hold unconditionally.
Let
The problem in this example is caused by the terms t and the patterns in all recursive calls of f on composed terms during the transformation of t are handled by the clauses of
This is formalized in the following two definitions.
A function specification
Note that the abstractions defined in Examples 4.10 and 4.12 are pattern-disjoint, while the one in Example 4.15 is not. Let
We define the uniform domain of
We will require that the protocol terms
(Substitution property).
Suppose that
We henceforth assume that
Equality preservation (P2)
Using the substitution property, we can reduce (P2) to the property stating that If
Neither of these properties holds in this generality (recall Remark 4.13). The following example illustrates a violation of (P2.a).
Let
To solve the problem described in Example 4.19, we introduce the notion of
Note that the abstraction
Let
To establish (P2.b) for a term t, we make use of the finite variant property of our rewrite theory.
We say that
For terms
(Equality preservation).
Suppose that
This concludes our treatment of (P2). We proceed with deducibility preservation.
Deducibility preservation (P1)
To preserve reachability and secrecy properties, our typed protocol abstractions need to preserve term deducibility, i.e., whenever a term t is deducible from a set of terms T then
(Preserving decryption).
Consider the composition rule
Generally speaking, we have to ensure that if a composed term
(Dropping fields).
Consider the derivation of rule
This example shows that we cannot drop fields from argument positions of a constructor that can be extracted by a rewrite rule (here decryption).
(Transforming non-enclosing constructors).
Suppose f transforms asymmetric encryptions and
The problem here is that the decryption rewrite rule is no longer applicable. This can be avoided by requiring that f is homomorphic for the constructors of the left-hand side l of the rewrite rule other than those enclosing the extracted term in l (here, the key constructors
(Non-linear variables).
This example illustrates another way to destroy the applicability of a rewrite rule by abstraction. Consider the rule
In this case, the problem is that the two instances of X in the rewrite rule are transformed differently, which destroys the matching. This suggests that if a constructor c enclosing the extracted term in l has a non-linear variable at its ith argument position then the equations of f must not split the ith argument of c.
The examples above (partly) motivate the following definitions.
We call a typed abstraction field-preserving for position i of c if, for all equations of non-splitting for position i of c if
Note that if
We say that a rewrite rule
For example, the projection rewrite rule
(Compatibility with rewrite theory).
A typed abstraction l has an arbitrary shape and either r is a constant,
We say that
We illustrate this definition with an example.
Let us check that the typed abstraction
Let us consider the symmetric decryption rule
Next, we verify that
We first establish a version of deducibility preservation without substitutions.
Let
By combining this theorem with Theorems 4.23 and 4.18, we can now derive (P1) which we formalize as the following corollary.
(Deducibility preservation with substitutions).
Let
This completes our discussion of (P1). Next, we discuss syntactic criteria for the disequality preservation in condition (iii) of Theorem 4.14.
Syntactic criteria for disequality preservation (P3)
Condition (iii) of Theorem 4.14 requires that, for all
Here, we present such a criterion that is applicable if t and u do not contain any message variables. Assuming that
Let
Note that for this criterion to be applicable, we require that f is the identity for the terms in positively occurring equations. This is often the case, as these terms typically have a simple structure, e.g., nonces or timestamps. However, this criterion cannot be used to justify the soundness of the typed abstraction from Example 4.12 with respect to the property
Typed abstractions offer a wide range of possibilities to transform cryptographic operations including subterm removal, splitting, and pulling fields outside of such an operation. We complement these abstractions with two kinds of untyped abstractions. The first type, discussed here, allows us to remove unprotected atoms and variables of any type. The second type removes redundancy in the form of intruder-derivable terms and is discussed in the next subsection.
Specification of atom-and-variable removal
We first present the formal definition of atom-and-variable removal abstractions, then we motivate some restrictions needed for soundness, and finally we illustrate the application of atom-variable removal on our running example.
An atom-and-variable removal abstraction does not remove all occurrences of an atom or a variable from a given term t, but those that are fields of t. Intuitively, these unprotected atoms and variables do not themselves provide any security properties and can therefore safely be removed. This intuition is most obvious for atom removal: the intruder already knows all constants and agent names and he can replace unprotected fresh values by his own ones of the same type. In the following definition, we formulate atom-and-variable removal abstracions, where we use the abbreviation
An atom-and-variable removal abstraction is a general abstraction
By point (i) any term in
Due to point (iii) of Definition 4.35, atom-and-variable removal abstractions cannot remove an atom or variable from a non-pair term. It is even unclear how to define this in general. Let us attempt to define a hypothetical variant
Consider the terms
Suppose that
Similar counterexamples can also be constructed if variable removal abstractions are considered. This highlights the necessity of point (iii) in Definition 4.35.
The following example shows that the soundness of
Consider terms
This example motivates the following definition.
A term u is clear in a term t if
Note that u is also clear in t if it does not appear at all in t. Our soundness result requires that all variables and atoms in T are clear in the terms to which
In the following example, we illustrate the use of atom-and-variable removal abstractions to transform We use atom-and-variable removal to simplify the protocol (
Soundness for atom-and-variable removal abstractions
We now turn our attention to the soundness result for atom-and-variable removal abstraction. This result requires that we restrict our attention to well-formed protocols. To define this predicate on protocols, we first introduce the notion of accessible variables.
(Accessible variables).
We say that a variable X is accessible in a term t if either
Intuitively, a variable X is accessible in a term t if there is a path from t’s root to an occurrence of X consisting of only extractable positions. This is to ensure that if X is accessible then it is potentially deducible. For example, X is accessible in
A protocol P is well-formed if all non-agent variables first occur in receive events, i.e., for all roles
A well-formed protocol captures the intuition that an agent must know what he sends and the elements that he receives into variables are accessible, e.g., by decrypting a ciphertext. Our notion of well-formedness is a weaker form of executability, which would additionally require that the agent also knows the relevant keying material. Hence, all practical protocols satisfy this condition.
Our soundness result for atom-and-variable removal abstractions is stated in the following theorem.
Let P be a well-formed protocol,
T is clear in
for all
Then for all states
To preserve attacks, condition (i) ensures that the removed atoms and variables are clear in all protocol terms. Condition (ii) requires that no removed atom or variable occurs in the property’s equalities. Together with condition (iii) it implies condition (a) of the definition of safe formulas (Definition 4.5). Condition (iv) requires that the initial knowledge of the intruder in the abstract protocol subsumes that in the original protocol. Finally, condition (v) reflects condition (e) of the definition of safe formulas (Definition 4.5).
We prove Theorem 4.43 by composing two separate soundness results for atom removal and for variable removal abstractions, respectively. Their statements and proofs appear in the full version [39].
Redundancy removal abstractions
The second kind of untyped abstractions are redundancy removal abstractions. A redundancy removal abstraction
Specification of redundancy removal abstractions
We now formally define our class of redundancy removal abstractions.
A redundancy removal abstraction for a protocol P is a general abstraction for all
Note that in these rules, for all terms
Intuitively, the predicate
In the following example, we illustrate the use of redundancy removal abstractions to further simplify the protocol First, we recall The 
The soundness result for redundancy removal abstractions is stated in the following theorem.
Let P be a protocol,
for all
(Soundness for redundancy removal abstractions).
Then for all states
Well-formedness preservation for protocol abstractions
In this section, we present well-formedness preservation results for our three types of protocol abstractions. These results are required for the composition of typed abstractions, atom-and-variable removal abstractions, and redundancy removal abstractions to transform well-formed protocols.
Let
Let T be a set of atoms and variables such that T is clear in
Let
The proofs of these propositions can be found in the full version [39].

The abstraction workflow for the analysis of security protocols.
Recall that our aim is to make protocol verification more efficient. Given a protocol and a property, our high-level idea is to construct a simpler version of the protocol and the property that is easier to verify. In particular, if the simpler version is a sound abstraction of the original, then we can conclude that the original also satisfies its property.
In the previous section, we gave sufficient conditions for abstractions to be sound. However, not all sound abstractions are useful for verification. In particular, if an abstraction is vulnerable to an attack that does not apply to the original, then we might waste verification time to find this attack, without being able to draw any conclusion about the original. Ideally, abstractions for verification extract the “core” of the cryptographic protocol, i.e., those parts of the protocol that are instrumental in achieving the property, and omit all other constructions. In this ideal case, the abstractions would have exactly the same properties as the original.
In this section, we describe an algorithm for efficient protocol verification based on such abstractions. Because we do not have a direct construction algorithm for sound abstractions, we use heuristics to generate reasonable abstractions and then check if they meet the soundness conditions. The workflow of our algorithm is described in Fig. 4: we first generate a stack of successively more abstract protocols and properties, with at the bottom the original, and at the top an abstract protocol that we hope represents the core of the protocol required to establish the property.
We then verify the protocols and the properties in this stack top-down, based on the assumption that it is more efficient to analyze a more abstract protocol. We provide empirical evidence for this in the next section. If we can successfully verify a protocol from the stack, we know the original protocol meets its property, and we can stop the analysis. If we find an attack, we try to reconstruct the attack on the original. If this is possible, we know the original protocol does not satisfy the property. If not, the attack is spurious, and we proceed to the next protocol on the stack, which is less abstract than the previous one.
We describe in Section 5.1 how we generate abstractions and in Section 5.2 how we check for spurious attacks.
Our heuristics to generate abstractions uses three strategies, corresponding to our three types of abstractions, which we apply in order. After applying a strategy, we check if the resulting abstraction is sound. We discuss the three strategies in turn.
Simplifying or removing constructors that might not be needed to establish the property

Structure of u.
Many protocols use (cryptographic) constructors that are, at most, needed to guarantee some (but not all) of its desired properties. To see this, consider the following example.
Let k be a session key and t an arbitrary term. Let u be defined as
If the security property encodes that t needs to be authenticated, we look for the strongest mechanism that could guarantee this. Within u, this would be the symmetric encryption with the long-term key
If the security property encodes that t needs to be secret, the situation changes, since secrecy needs to be guaranteed for all occurrences of t, and not just one. Thus, in the left branch, secrecy of t is guaranteed on the basis of the session key k, whereas in the right branch, secrecy is guaranteed on the basis of both constructors. Thus, within u, t’s secrecy depends on the secrecy of another term, and not just the long-term key. When we want to abstract the term u sent in a protocol without introducing new attacks, we need to ensure we do not make the situation worse. Thus, we would not modify the left branch. However, in the right branch we could remove t from the protection of either one of the constructors, since the overall guarantee within u would still be the same.
We will exploit this intuition by first determining which (sub)terms are relevant for establishing the desired property. We represent this by assigning security labels to each of them. In a second step, we give an algorithm that moves subterms out of their encapsulating constructor as long as their security labels are not increased.
For the first step, we first define which constructors serve which purpose. For example, a hash function does not authenticate its subterms, but it does not reveal its subterms either, and hence may be used in the context of secrecy. We differentiate between two main objectives (authentication and confidentiality) and assign one of three labels for each.
Security labels. We define the set of (security) labels
The labels for constructors (i.e., the guarantees they establish for their subterms) are specified by the functions
We define an auxiliary function
Security labels for different cryptographic operations, encoding what they might achieve for their strict subterms
Security labels for different cryptographic operations, encoding what they might achieve for their strict subterms
Let P be a protocol, ϕ a property, and t a term. We define the protocol authentication label
For confidentiality, we cannot take the maximum over all positions, since we need to ensure that all occurrences of t are protected. Thus, we consider the labels of all paths on which t occurs, and take the minimum.
(Protocol confidentiality label).
Let P be a protocol, ϕ a property, and t a term. We define the protocol authentication label
Let us consider the terms u and t in Fig. 5. Suppose that
We use the label definitions to construct an abstraction in the following way. First, we compute the authentication and confidentiality labels for all terms in the protocol and property. Second, we construct candidate abstractions in which we pull subterms out of their constructors (e.g., abstracting
for each term, the labels in the candidate abstraction are not lower than those of the corresponding terms in the original.
In many cases, there are atoms or variables that occur in the protocol messages but that do not occur in the security property ϕ. They might therefore be redundant and we generate an abstraction in which they are removed from the protocol messages. Such simplifications can be achieved by atom-and-variable removal abstractions. In the full version [39], we present an algorithm that identifies unnecessary atoms and variables, and removes them from the protocol messages.
Removing redundant terms based on preceding intruder knowledge
A somewhat related case occurs for terms in a protocol message m that the intruder can derive from his previous knowledge. A sufficient condition for this is that they can be derived from the combination of the initial intruder knowledge and the messages sent before m in the same role. As before, they might be redundant and we generate an abstraction in which they are removed from the protocol messages. In the full version [39], we explain how to eliminate such redundancies using redundancy removal abstractions.
Checking for spurious attacks
Our abstractions are sound, but not complete. Therefore, we may encounter false negatives, i.e., spurious attacks. To check whether an attack on a security property ϕ in an abstract model corresponds to a real attack in the original one, we perform the following steps. First, for each thread in the attack trace, we construct a (symbolic) trace whose events correspond to those occurring in the abstract thread. Then, we ask the verifier to search for an attack in the original protocol such that this attack contains only threads that are computed in the previous step. Formally, let
Implementation and case studies
In this section, we explain how we have implemented our abstraction mechanism for the Scyther tool. The resulting tool is available online [36]. We then validate the effectiveness of our method on a large number of real-world case studies.
Implementation for the Scyther tool
Scyther [14] is a leading automated security protocol verification tool. It supports verification for both a bounded and an unbounded number of threads. It also supports multi-protocol analysis, i.e., verifying a composition of multiple protocols. Scyther takes as input a security protocol description specified by a set of linear role scripts, which include the intended security properties. The tool supports both user-defined types and hash functions. These features match our setting very well.
In this section, we first present the correspondence between claim events in Scyther and our security property formulas. Then, we describe an extension of the labeling mechanism and the abstraction heuristics. In the full version [39], we demonstrate the application of our abstraction heuristics on an example.
Claim events and security properties
In Scyther, security properties are specified by means of claim events, which are integrated into protocol role specifications. Intuitively, claim events express the intended security goal that an agent executing a given protocol role expects to achieve. For our implementation, we consider the following types of claim events that are used to express secrecy and various forms of authentication properties. We adopt the definitions of these properties from [13,14,32]. All these properties include the additional premise that both the agent owning the thread executing the claim and its (intended) communication partner are honest, which we do not repeat below.
Note that this property still holds even when b was running the protocol with someone else (not a). Strengthening aliveness leads us to the notion of weak agreement property.
Neither aliveness nor weak agreement guarantee that agents agree on their respective roles or on any data exchanged. This additional requirement is captured by non-injective agreement.
Note that non-injective agreement specified by
We now explain how to formalize these properties in our security property language using an example. Consider the Needham–Schroeder public-key (NSPK) protocol from [35]. We mimic the claim events by introducing the corresponding signal events with the following set of signals:
The last two properties are obtained by instantiating the general definitions from [13] for the A role of the Needham–Schroeder public-key protocol. To see that
In practice, it turns out that the labeling mechanism previously described is not sufficient to achieve good abstractions. There are protocols that employ cryptographic primitives in particular ways to achieve certain security goals, even though these primitives do not provide the desired properties themselves. In such cases, the heuristic may assign security labels to terms incorrectly, or accidentally remove elements that are important to achieve these properties.
Let us come back to the NSPK protocol, specified (without signals) as:
To deal with this issue, we enable the heuristic to detect such a pattern, i.e., an asymmetric encryption that includes an agent identity which is different from the one indicated in the encryption key. In this case, at least one occurrence of the identity must be kept, and the encryption is associated with authentication label
We have validated the effectiveness of our abstractions on a total of 24 members of the IKE and ISO/IEC 9798 protocol families and on the PANA-AKA protocol [4] and the KSL protocol. We verify these protocols using five tools based on four different techniques: Scyther [14], CL-Atse [47], OFMC [8], SATMC [6], and ProVerif [9]. Only Scyther and ProVerif support verification of an unbounded number of threads. In Table 3, we present a selection of the experimental results for Scyther and refer to the full version [39] for a complete account, including results for the other tools for which we used hand-crafted abstractions. While our execution model closely fits Scyther’s, there are subtle differences with the execution models and specification languages of the other tools. However, our initial results suggest that our techniques can be formally adapted to increase the efficiency of those tools as well. Our models of the IKE and ISO/IEC 9798 protocols are based on Cremers’ [11,12]. Since Scyther uses a fixed signature with standard cryptographic primitives and no equational theories, the IKE models approximate the DH equational theory by oracle roles.
For our case studies, we verify several security properties including secrecy, aliveness, weak agreement, and non-injective agreement. We mark verified properties by ✓ and falsified ones by ×. An entry
Experimental results for Scyther. The time is in seconds. No : Number of abstractions. Properties: S ecrecy, A liveness, W eak agreement, and N on-injective agreement
Experimental results for Scyther. The time is in seconds.
Verification. For 13 of the 19 original protocols that are analyzed, an unbounded verification attempt results in a timeout (TO = 8h cpu time) or memory exhaustion (ME). In 7 of these, our abstractions enabled the verification of all properties in less than 0.4 seconds and in one case in 78 seconds. However, for the first three protocols, we still get a timeout. For the large majority of the bounded verification tasks, we significantly push the bound on the number of threads and achieve massive speedups. For example, our abstractions enable the verification of the complex nested protocols IKEv2-eap and PANA-AKA. Scyther verifies an abstraction of IKEv2-eap for up to 6 threads and, more strikingly, completes the unbounded verification of the simplified PANA-AKA in under 0.3 seconds whereas it can handle only 4 threads of the original version.
For these protocols, our tool aggressively simplifies the original models by removing unnecessary cryptographic protections and redundant fields. The IKEv2-eap protocol consists of two roles exchanging 8 messages. The messages are large and contain up to 5 layers of cryptographic operations (such as encryptions, signatures, and hashes). However, the most abstract model generated by our tool only exchanges 5 messages (i.e., 3 messages are completely removed by untyped abstractions). The most deeply nested messages contain only 3 layers of cryptographic operations. The PANA-AKA protocol exhibits a similar complexity. It employs up to 6 layers of cryptographic operations. Even though the most abstract model for PANA-AKA still exchanges 7 messages, the messages are substantially smaller than those of the original model and use at most 3 layers of cryptographic operations. We also achieve dramatic speedups for many other protocols, most notably for IKEv1-pk-a22, ISO/IEC 9798-2-6, and ISO/IEC 9798-3-6-2. This shows that our abstractions work particularly well for protocols that have complex message structures or large numbers of exchanged messages, as these features can significantly deteriorate the performance of protocol verifiers.
More interestingly, our abstractions also perform very well on another class of protocols which have simple message structures but still render verification challenging. For example, the ISO/IEC 9798-3-6-1, ISO/IEC 9798-3-7-1 and KSL protocols contain relatively small messages with at most one layer of encryption. However, the verification attempts for the original versions of the ISO/IEC 9798-3-6-1 and ISO/IEC 9798-3-7-1 protocols both result in memory exhaustion after 7 threads. Similarly, the verification of KSL already times out for 5 threads. We attribute this difficulty to the presence of untyped variables, i.e., variables of type
Apart from enormous performance gains, the speedup is more modest for a few protocols, e.g., IKEv1-pk2-a2, IKEv2-sigtomac, and IKEv2-mac. These protocols have simple message structures, e.g., using at most 3 layers of cryptographic operations and only up to 4 exchanged messages. Moreover, they use untyped variables only in protected positions, i.e., as arguments of a hash or an encryption. They therefore do not leave much room for abstractions. In fact, although the generated abstract models for these protocols have smaller message sizes, they have similar message structures compared to the original ones. Nevertheless, our abstractions enable the reduction of the verification time by an order of magnitude in some cases, e.g., for the IKEv1-pk2-a2 protocol.
Additionally, we observe that the verification time for many abstracted protocols increases much more slowly than for their originals as the number of threads increases. We obtain almost constant verification times for the six ISO/IEC 9798 protocols, whereas the time significantly increases on some originals, e.g., for the ISO/IEC 9798-3-6-1 protocol.
Falsification. For rows marked by ×, the second line corresponds to falsification time for the most abstract model, which is much faster than on the original one. For example, for 8 threads of the IKEv1-pk-m protocol, we reduce falsification time from a timeout to 2.05 seconds. Note that for falsification, a check for spurious attacks is needed. This subroutine renders the performance gains less substantial than that for verification. For instance, in the unbounded case, the speedup factors are 1.15 for IKEv1-sig-m and 4.19 for IKEv1-sig-m-perlman. Note that our tool automatically checks for spurious attacks. Interestingly, all attacks found in the most abstract protocols are real, suggesting that our measures to prevent spurious attacks are effective.
Combination. For the IKEv1-pk-m2 and IKEv2-sig-child protocols, the tool verifies non-injective agreement for one role and falsifies it for the other one. Analogous to other case studies, we obtain a remarkable speedup for these protocols. Our abstractions raise the feasibility bound by 2 to 3 additional threads.
Hui and Lowe [28] define several kinds of abstractions similar to ours with the aim of improving the performance of the CASPER/FDR verifier. They establish soundness only for ground messages and encryption with atomic keys. We work in a more general model, cover additional properties, and treat the non-trivial issue of abstracting the open terms in protocol specifications. Other works [17,18,41] also propose a set of syntactic transformations, however without formally establishing their soundness. Using our results, we can, for instance, justify the soundness of the refinements in [18, Section 3.3].
Backes et al. [7] study the abstraction of authentication protocols formalized in the ρ-spi calculus. They propose a static analysis for authentication protocols by abstracting challenge-response messages into non-cryptographic versions expressed in a different language, called the CR calculus. Their abstraction method is based on non-increasing security labels similar to those of our heuristics. However, there are several differences with our work. First, since their sound abstractions map protocol specifications to a different language, the abstract protocols cannot be further abstracted. In our setting, protocol specifications and abstract protocols are expressed in the same language and abstractions can be composed. Second, the construction of the abstractions requires the identification of challenge-response components of a protocol, for which they do not give an algorithm. Third, since they designed a specialized technique for proving authentication properties, they cannot employ existing protocol verification tools to verify the abstract protocols. In contrast, our abstractions are composable, computed automatically by our tool, and can be verified using standard protocol verifiers. Finally, their method is restricted to agreement properties, while ours supports an expressive property specification language, which covers secrecy and a variety of authentication properties.
Guttman [24 ,25] studies the preservation of security properties for a rich class of protocol transformations in the strand space model. His approach to property preservation is based on the simulation of protocol analysis steps instead of execution steps. Each such analysis step explains the origin of a message. Apart from this different approach to soundness, there are other differences with our work. First, instead of working at the level of protocol messages, his protocol transformations are applied to strand space nodes and then lifted to protocol specifications and security properties. In contrast to our work, his approach does not restrict the shape of the transformed protocol message with respect to the original message. In his theory, one can, for instance, transform a hash of a message X and a key K into an encryption of X with K. We do not support such general transformations. Second, his protocol transformations are required to preserve the origination of values and the plaintext subterms of messages. The former condition means that if a value x first occurs in a transmission node then it also occurs first in the corresponding transformed node. Our soundness results do not require such conditions. For example, we can completely remove fresh values that are in clear or fields in a hash. Third, since his primary focus was to set up a general framework to express and justify security protocol transformations, he does not provide syntactic soundness conditions, guidance for the choice of appropriate abstractions, or automated verification. It might be possible to identify a subset of his transformations for which this is possible, but this would require additional work. In contrast, our tool automatically determines suitable abstractions and checks their soundness.
Refinement is abstraction viewed in the reverse direction, i.e., from abstract to concrete. Sprenger et al. [31,45,46] have proposed a hierarchical development method for security protocols based on stepwise refinement that spans several levels of abstraction. Each development starts from abstract models of security properties and proceeds down to cryptographic protocols secure against a Dolev–Yao intruder. The development process traverses intermediate levels of abstraction based on message-less protocols and communication channels with authenticity and confidentiality properties. Security properties, once proved for a given model, are preserved by further refinements. They have applied their method to develop families of authentication and key transport protocols. The abstractions in the present paper belong to their most concrete level of cryptographic protocols. They have embedded their approach in the Isabelle/HOL theorem prover, but each refinement step essentially requires a separate soundness proof.
Conclusions
In this work, we propose a set of syntactic protocol transformations that allows us to abstract realistic protocols and capture a large class of attacks. Unlike previous work [28,37], our theory and soundness results accommodate equational theories and a fine-grained type system that supports untyped variables, user-defined types, and subtyping. These features allow us to accurately model protocols, capture type-flaw attacks, and adapt to different verification tools, e.g., those supporting equational theories such as ProVerif and CL-Atse. We have extended Scyther with an abstraction module, which we validated on various IKE and ISO/IEC 9798 protocols and others. We also tested our technique (with manually produced abstractions) on ProVerif, CL-Atse, OFMC, and SATMC. Our experiments show that modern protocol verifiers can substantially benefit from our abstractions, which often either enable previously infeasible verification tasks or lead to dramatic speedups. Our abstraction tool supports checking for spurious attacks, which allows us to not only verify but also falsify security protocols efficiently.
As for future work, we plan to extend our soundness results to more expressive security protocol models such as multiset rewriting. This would allow us to cover more security protocols, for instance, protocols involving loops such as the TESLA protocol [42] or non-monotonic states such as contract signing protocols [3], as well as more security properties and adversary capabilities such as perfect forward secrecy, key compromise impersonation, and adversaries capable of revealing the local state of agents. We believe that our soundness results can also be extended to support else-branches in such theories by additionally establishing preservation theorems for disequality tests. Another direction for future research could be to generalize the tool and support more protocol verifiers. Possible improvements might be gained from applying techniques from the field of counter-example guided refinement: when a spurious attack is found, it might be possible to extract information from it to guide the exploration of the generated abstractions.
Footnotes
Acknowledgments
We thank Mathieu Turuani and Michael Rusinowitch for our fruitful technical discussions on the topic of this paper. We are also grateful to David Basin, Ognjen Maric, Ralf Sasse, and the anonymous reviewers for their careful proof-reading and helpful suggestions. This work was partially supported by the Air Force Office of Scientific Research, grant number FA9550-17-1-0206, and the EU FP7-ICT-2009 Project No. 256980, NESSoS: Network of Excellence on Engineering Secure Future Internet Software Services and Systems.
