Interface Guidance

  • All Known Subinterfaces:
    DiffFuzzGuidance
    All Known Implementing Classes:
    AFLGuidance, DiffFuzzNoGuidance, DiffFuzzReproGuidance, ExecutionIndexingGuidance, NoGuidance, PerfFuzzGuidance, ReproGuidance, ReproServerGuidance, ZestGuidance

    public interface Guidance
    A front-end for guided fuzzing.

    Before each fuzzing trial, the front-end is queried for input to be used as a source of random numbers by junit-quickcheck's generators.

    During a fuzzing trial, trace events are generated, and the front-end provides callbacks to handle these trace events (e.g. to collect branch coverage).

    At the end of a fuzzing trial, the front-end is notified of the result of the trial.

    The standard sequence of method invocations on a Guidance instance is as follows (in pseudo-code):

         while (guidance.hasInput()) {
             Random rng = new StreamBackedRandom(guidance.getInput());
             Object[] args = generateInput(rng);
             try {
                 runTest(args); // generates many trace events
                 guidance.handleResult(SUCCESS, null);
             } catch (AssumptionViolatedException e) {
                 guidance.handleResult(INVALID, e);
             } catch (Throwable t) {
                 if (isExpected(e)) {
                     guidance.handleResult(SUCCESS, e);
                 } else {
                     guidance.handleResult(FAILURE, e);
                 }
             }
         }
     
    See the implementation of FuzzStatement for the real loop.
    • Method Detail

      • getInput

        InputStream getInput()
                      throws IllegalStateException,
                             GuidanceException
        Returns a reference to a stream of values return from the pseudo-random number generator.

        This method is guaranteed to be invoked by JQF at most once after each invocation of hasInput() that has returned true.

        If hasInput() returns false or has not been invoked since the last call to getInput(), then invoking this method may throw an IllegalStateException.

        Returns:
        a stream of bytes to be used by the input generator(s)
        Throws:
        IllegalStateException - if the last hasInput() returned false
        GuidanceException - if there was an I/O or other error in generating the input stream
      • hasInput

        boolean hasInput()
        Returns whether an input is ready for a new trial to be executed.

        This method may block until a new input stream is made ready. If this method returns false, then JQF stops fuzzing and this guidance will not be used further.

        Returns:
        whether a new trial should be executed
      • observeGeneratedArgs

        default void observeGeneratedArgs​(Object[] args)
        Callback for observing actual arguments passed to the test method.

        This method is invoked exactly once after each call to getInput(). The arguments to this callback are the structured inputs that are produced by junit-quickcheck generators, which in turn decode the bytes produced by getInput().

        This method is useful for logging the generated args or for calculating the size of the generated args. The default implementation does nothing.

        Parameters:
        args - an array of arguments that will be passed to the test method; the size of this array is equal to the number of formal parameters to the test method
      • handleResult

        void handleResult​(Result result,
                          Throwable error)
                   throws GuidanceException
        Handles the end of a fuzzing trial.

        This method is guaranteed to be invoked by JQF exactly once after each invocation of getInput(). Therefore, it is safe to open resources such as files in a call to getInput() and only close them inside this method.

        If result is SUCCESS, then error is either null or it is an instance of a throwable that is declared by the test method in its throws clause.

        If result is INVALID, then error is either an AssumptionViolatedException, if the argument of an assume() statement was false, or it is a GuidanceException, indicating that fuzzing was interrupted during the execution of this trial (and will not continue further).

        If result is FAILURE, then error is some other throwable that was thrown by the test execution but was not listed in its throws clause. This is the only way to detect test errors. Assertion errors will typically fall into this category. So will other unlisted RuntimeExceptions such as NPE.

        Parameters:
        result - the result of the fuzzing trial
        error - the error thrown during the trial, or null
        Throws:
        GuidanceException - if there was an I/O or other error in handling the result
      • generateCallBack

        Consumer<TraceEvent> generateCallBack​(Thread thread)
        Returns a callback generator for a thread's event trace.

        The application under test is instrumented such that each thread generates a sequence of TraceEvents that may be handled by a separate callback method (though it may also be the same callback).

        The callback provided by this method will typically be used for collection execution information such as branch coverage, which in turn is used for constructing the next input stream.

        This method is a supplier of event consumers. It is invoked once per new application thread spawned during fuzzing.

        Parameters:
        thread - the thread whose events to handle
        Returns:
        a callback that handles trace events generated by that thread
      • run

        default void run​(TestClass testClass,
                         FrameworkMethod method,
                         Object[] args)
                  throws Throwable
        Runs a test method with generated arguments as input.

        By default, this method simply runs the test method using a JUnit TrialRunner. Guidances can override this method to customize how test execution should be performed once inputs are generated. For example, a guidance that supports non-deterministic test code may wish to execute multiple trials per generated input.

        Parameters:
        testClass - the test class
        method - the test method within the test class
        args - the arguments to the test method (i.e., the test input)
        Throws:
        Throwable - any exception that may be thrown during test execution