OpenAPI Annotation Design

General Design

Background

Existing Problems and Challenges:

  1. API Coverage:

    • Some parameters are almost impossible to generate randomly

    • Some API operations are critical in terms of logic (login, checkout, etc.), and cannot be arbitrarily called.

  2. Resources are created on each run with very low efficiency

  3. Multi-API vulnerabilities are not considered and discussed.

Goal: Annotate the original OpenAPI specification to guide the API fuzzing process, so that

  • Multi-API vulnerabilities are addressed

  • Resources are efficiently reused

Basic Idea: Annotation on each parameter and each operation

Parameter Annotation: label value rendering strategy

  • Group 1: parameter value property

    • Fixed: use example list
    • Dynamic: use the success result from previous calls in one testcase
    • History: use history value list
    • Mutation: mutate based on the example list or history value to improve coverage
    • Random: a random generation strategy

  • Group 2: parameter source

    • User-controlled
    • System generated

Operation Annotation: label each operation

  • Group 1: CRUD property
    • pre-operations: operations that is required before others. Example: login, key generation, etc.
    • post-operations: operations that is required after others. Doing this will change the whole service status. Example: logout, change password, checkout, etc.
    • Switch: a switch button that will change service logic
    • CREATE/UPDATE- READ/PROCESS - DELETE
  • Group 2: processed parameter

Implementation with OpenAPI 3.0

Our idea is to utilize the “Specification Extensions“ concept (https://spec.openapis.org/oas/v3.1.0#specification-extensions) supported by OpenAPI 3.0 to achieve this. The general approach is to create a “x-annotation“ field as an additional property for each object. We may want to have a better name for it now. Below we give the example of how to annotate a piece of SpreeCommerce OpenAPI 3.0 specification with our design.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
paths:
  /api/v2/storefront/account:
    post:
      description: reates a new account. This endpoint requires [Spree Auth Devise](https://github.com/spree/spree_auth_devise) gem installed. 
      summary: Create an Account
      x-operation-annotation: {"CRUD": ["pre-operation", "CREATE"], "parameters": ["user.email", "user.password", "user.password_confirmation"]}
      tags:
        - Account
      operationId: Account Creation
      requestBody:
        required: true
        content:
          application/vnd.api+json:
            schema:
              type: object
              properties:
                user:
                  type: object
                  properties:
                    email:
                      type: string
                      example: john@snow.org
                      x-param-annotation: {"property": ["Fixed"], "source":"user-controlled"}
                    password:
                      type: string
                      example: spree123
                      x-param-annotation: {"property": ["Fixed"], "source":"user-controlled"}
                    password_confirmation:
                      type: string
                      example: spree123
                      x-param-annotation: {"property": ["Fixed"], "source":"user-controlled"}
                                          
      responses: ...

Such documentation can be directly taken by OpenAPI Swagger without any problems, while allows a better interpretation in API Testing process.

Initialization

*Parameter annotation: *

  • Group 1:
    • Use keywords: for instance, parameter like username, password etc. are labeld with fixed
    • Manual: manually label the others
  • Group 2:
    • After graph construction, group 2 property can be auto labeled.

Operation annotation

  • Group 1:
    • Manual: manually label the key operations (login, etc.)
    • The rest can be labeled based on HTTP request method.
  • Group 2:
    • Auto labeled. One operation can have multiple labels

Guided Fuzzing

General Strategy Description

Our strategy contains two rounds of execution feedback procedures. (1) It refines the generated \graph{} with an exploration strategy that tries to maximize the endpoints coverage. (2) It focuses on resources reuse to identify potential single/multi API vulnerabilities.

Parameter Generation Strategy

The annotation for each parameter is parsed by \tool{} and actively maintained during the fuzzing process. Each parameter is generated based on its annotation strategy in the following sequence.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
#1: Group2 -> user-controlled vs. system-generated
    if system-generated:
        strategy = dyanmic
#2: Group1 -> dynamic vs. other strategy
    if dynamic:
        track_param_value_history()
    else:
        if fixed:
            use_dictionary()
        if mutation:
            value = use_dictionary + use_history
            mutate(value)
        if random:
            random_generation(datatype)

Sequence Generation Strategy

We explain the two strategies

Strategy 1: Explore

This strategy generates all possible sequences by recursively visiting each parameter and concatenate the prior/posterior operations. Note that the pre/post operations are excluded from this step.

For testcase generation:

  1. Append necessary prior operations to each call sequence.
  2. Enumerate all the call sequences
  3. For each call sequence, render the parameters for each request in order and execute one by one. Note that for each call sequence, a full CRUD cycle is executed, and parameter values are never repeated if possible.

Strategy 2:

This strategy tries to maximize resource utilization so that it will efficiently identify possible vulnerabilities in target call sequences.

  1. It selects a user-controlled parameter that can be investigated.
  2. From the parameter, it generates all the possible call sequences for this parameter.
  3. Use CRUD filter to select those sequences where the parameter is in CREATE or UPDATE.

For testcase generation:

  1. Render the parameter values until the injection point. We consider the previous resources are “generated” and fixed, and repeatedly execute the rest of the sequences for vulnerability identification.
  2. Use mutation strategy to embed payloads into the injection target parameter.
  3. For each injection, execute until the end of the request sequence, excluding DELETE.
  4. Use after free is a special type of vulnerability. We will only test that after each round of testing to maximize resource utilization. (There are use-after-free type of API vulnerabilities, but very rare and we didn’t see that in our cases.)

Dynamic Updates

Parameters

During the test procedure, the annotations are dynamically updated to improve testing performance. The general idea is to use status code to reflect if a parameter is properly generated. The detailed strategies are explained below.

  1. Save all the 20x request parameter values into history database. The annotation for parameters with history is changed to mutation. (Consider to count success, and change strategy to fully random when possible?.)
  2. For each 40x request, at least one parameter value is not correctly rendered. The algorithm will change all random annotation to mutation, and one mutation annotation to history.
  3. For each 50x request, report as a bug. In this work, we do not focus on the reason of the bugs since they may not have security impacts.

Specially, when several parameters share the same value history, the graph will be refined and their source property may be changed.

Operations

The label for operations are generally not updated.