Class ZestGuidance

  • All Implemented Interfaces:
    Guidance
    Direct Known Subclasses:
    ExecutionIndexingGuidance

    public class ZestGuidance
    extends Object
    implements Guidance
    A guidance that performs coverage-guided fuzzing using two coverage maps, one for all inputs and one for valid inputs only.
    Author:
    Rohan Padhye
    • Field Detail

      • random

        protected Random random
        A pseudo-random number generator for generating fresh values.
      • testName

        protected final String testName
        The name of the test for display purposes.
      • maxDurationMillis

        protected final long maxDurationMillis
        The max amount of time to run for, in milli-seconds
      • maxTrials

        protected final long maxTrials
        The max number of trials to run
      • numTrials

        protected long numTrials
        The number of trials completed.
      • numValid

        protected long numValid
        The number of valid inputs.
      • outputDirectory

        protected final File outputDirectory
        The directory where fuzzing results are produced.
      • savedCorpusDirectory

        protected File savedCorpusDirectory
        The directory where interesting inputs are saved.
      • savedFailuresDirectory

        protected File savedFailuresDirectory
        The directory where saved inputs are saved.
      • allInputsDirectory

        protected File allInputsDirectory
        The directory where all generated inputs are logged in sub-directories (if enabled).
      • currentInput

        protected ZestGuidance.Input<?> currentInput
        Current input that's running -- valid after getInput() and before handleResult().
      • currentParentInputIdx

        protected int currentParentInputIdx
        Index of currentInput in the savedInputs -- valid after seeds are processed (OK if this is inaccurate).
      • numChildrenGeneratedForCurrentParentInput

        protected int numChildrenGeneratedForCurrentParentInput
        Number of mutated inputs generated from currentInput.
      • cyclesCompleted

        protected int cyclesCompleted
        Number of cycles completed (i.e. how many times we've reset currentParentInputIdx to 0.
      • numFavoredLastCycle

        protected int numFavoredLastCycle
        Number of favored inputs in the last cycle.
      • blind

        protected boolean blind
        Blind fuzzing -- if true then the queue is always empty.
      • validityFuzzing

        protected boolean validityFuzzing
        Validity fuzzing -- if true then save valid inputs that increase valid coverage
      • numSavedInputs

        protected int numSavedInputs
        Number of saved inputs. This is usually the same as savedInputs.size(), but we do not really save inputs in TOTALLY_RANDOM mode.
      • runCoverage

        protected ICoverage runCoverage
        Coverage statistics for a single run.
      • totalCoverage

        protected ICoverage totalCoverage
        Cumulative coverage statistics.
      • validCoverage

        protected ICoverage validCoverage
        Cumulative coverage for valid inputs.
      • maxCoverage

        protected int maxCoverage
        The maximum number of keys covered by any single input found so far.
      • responsibleInputs

        protected Map<Object,​ZestGuidance.Input> responsibleInputs
        A mapping of coverage keys to inputs that are responsible for them.
      • uniqueFailures

        protected Set<String> uniqueFailures
        The set of unique failures found so far.
      • EXACT_CRASH_PATH

        protected final String EXACT_CRASH_PATH
        save crash to specific location (should be used with EXIT_ON_CRASH)
      • verbose

        protected final boolean verbose
        Whether to print log statements to stderr (debug option; manually edit).
        See Also:
        Constant Field Values
      • console

        protected final Console console
        A system console, which is non-null only if STDOUT is a console.
      • startTime

        protected final Date startTime
        Time since this guidance instance was created.
      • lastRefreshTime

        protected Date lastRefreshTime
        Time at last stats refresh.
      • lastNumTrials

        protected long lastNumTrials
        Total execs at last stats refresh.
      • STATS_REFRESH_TIME_PERIOD

        protected final long STATS_REFRESH_TIME_PERIOD
        Minimum amount of time (in millis) between two stats refreshes.
        See Also:
        Constant Field Values
      • logFile

        protected File logFile
        The file where log data is written.
      • statsFile

        protected File statsFile
        The file where saved plot data is written.
      • currentInputFile

        protected File currentInputFile
        The currently executing input (for debugging purposes).
      • coverageFile

        protected File coverageFile
        The file contianing the coverage information
      • LIBFUZZER_COMPAT_OUTPUT

        protected final boolean LIBFUZZER_COMPAT_OUTPUT
        Use libFuzzer like output instead of AFL like stats screen (https://llvm.org/docs/LibFuzzer.html#output)
      • QUIET_MODE

        protected final boolean QUIET_MODE
        Whether to hide fuzzing statistics
      • LOG_ALL_INPUTS

        protected final boolean LOG_ALL_INPUTS
        Whether to store all generated inputs to disk (can get slowww!)
      • singleRunTimeoutMillis

        protected long singleRunTimeoutMillis
        Timeout for an individual run.
      • runStart

        protected Date runStart
        Date when last run was started.
      • branchCount

        protected long branchCount
        Number of conditional jumps since last run was started.
      • EXIT_ON_CRASH

        protected final boolean EXIT_ON_CRASH
        Whether to stop/exit once a crash is found.
      • firstThread

        protected Thread firstThread
        The first thread in the application, which usually runs the test method.
      • multiThreaded

        protected boolean multiThreaded
        Whether the application has more than one thread running coverage-instrumented code
      • SAVE_ONLY_VALID

        protected final boolean SAVE_ONLY_VALID
        Whether to save only valid inputs
      • MAX_INPUT_SIZE

        protected final int MAX_INPUT_SIZE
        Max input size to generate.
      • GENERATE_EOF_WHEN_OUT

        protected final boolean GENERATE_EOF_WHEN_OUT
        Whether to generate EOFs when we run out of bytes in the input, instead of randomly generating new bytes.
      • NUM_CHILDREN_BASELINE

        protected final int NUM_CHILDREN_BASELINE
        Baseline number of mutated children to produce from a given parent input.
        See Also:
        Constant Field Values
      • NUM_CHILDREN_MULTIPLIER_FAVORED

        protected final int NUM_CHILDREN_MULTIPLIER_FAVORED
        Multiplication factor for number of children to produce for favored inputs.
        See Also:
        Constant Field Values
      • MEAN_MUTATION_COUNT

        protected final double MEAN_MUTATION_COUNT
        Mean number of mutations to perform in each round.
        See Also:
        Constant Field Values
      • MEAN_MUTATION_SIZE

        protected final double MEAN_MUTATION_SIZE
        Mean number of contiguous bytes to mutate in each mutation.
        See Also:
        Constant Field Values
      • DISABLE_SAVE_NEW_COUNTS

        protected final boolean DISABLE_SAVE_NEW_COUNTS
        Whether to save inputs that only add new coverage bits (but no new responsibilities).
      • STEAL_RESPONSIBILITY

        protected final boolean STEAL_RESPONSIBILITY
        Whether to steal responsibility from old inputs (this increases computation cost).
    • Constructor Detail

      • ZestGuidance

        public ZestGuidance​(String testName,
                            Duration duration,
                            Long trials,
                            File outputDirectory,
                            Random sourceOfRandomness)
                     throws IOException
        Creates a new Zest guidance instance with optional duration, optional trial limit, and possibly deterministic PRNG.
        Parameters:
        testName - the name of test to display on the status screen
        duration - the amount of time to run fuzzing for, where null indicates unlimited time.
        trials - the number of trials for which to run fuzzing, where null indicates unlimited trials.
        outputDirectory - the directory where fuzzing results will be written
        sourceOfRandomness - a pseudo-random number generator
        Throws:
        IOException - if the output directory could not be prepared
      • ZestGuidance

        public ZestGuidance​(String testName,
                            Duration duration,
                            Long trials,
                            File outputDirectory,
                            File[] seedInputFiles,
                            Random sourceOfRandomness)
                     throws IOException
        Creates a new Zest guidance instance with seed input files and optional duration, optional trial limit, an possibly deterministic PRNG.
        Parameters:
        testName - the name of test to display on the status screen
        duration - the amount of time to run fuzzing for, where null indicates unlimited time.
        trials - the number of trials for which to run fuzzing, where null indicates unlimited trials.
        outputDirectory - the directory where fuzzing results will be written
        seedInputFiles - one or more input files to be used as initial inputs
        sourceOfRandomness - a pseudo-random number generator
        Throws:
        IOException - if the output directory could not be prepared
      • ZestGuidance

        public ZestGuidance​(String testName,
                            Duration duration,
                            Long trials,
                            File outputDirectory,
                            File seedInputDir,
                            Random sourceOfRandomness)
                     throws IOException
        Creates a new Zest guidance instance with seed input directory and optional duration, optional trial limit, an possibly deterministic PRNG.
        Parameters:
        testName - the name of test to display on the status screen
        duration - the amount of time to run fuzzing for, where null indicates unlimited time.
        trials - the number of trials for which to run fuzzing, where null indicates unlimited trials.
        outputDirectory - the directory where fuzzing results will be written
        seedInputDir - the directory containing one or more input files to be used as initial inputs
        sourceOfRandomness - a pseudo-random number generator
        Throws:
        IOException - if the output directory could not be prepared
      • ZestGuidance

        public ZestGuidance​(String testName,
                            Duration duration,
                            File outputDirectory,
                            File seedInputDir)
                     throws IOException
        Creates a new Zest guidance instance with seed inputs and optional duration.
        Parameters:
        testName - the name of test to display on the status screen
        duration - the amount of time to run fuzzing for, where null indicates unlimited time.
        outputDirectory - the directory where fuzzing results will be written
        seedInputDir - the directory containing one or more input files to be used as initial inputs
        Throws:
        IOException - if the output directory could not be prepared
      • ZestGuidance

        public ZestGuidance​(String testName,
                            Duration duration,
                            File outputDirectory)
                     throws IOException
        Creates a new Zest guidance instance with seed inputs and optional duration.
        Parameters:
        testName - the name of test to display on the status screen
        duration - the amount of time to run fuzzing for, where null indicates unlimited time.
        outputDirectory - the directory where fuzzing results will be written
        Throws:
        IOException - if the output directory could not be prepared
      • ZestGuidance

        public ZestGuidance​(String testName,
                            Duration duration,
                            File outputDirectory,
                            File[] seedFiles)
                     throws IOException
        Creates a new Zest guidance instance with seed inputs and optional duration.
        Parameters:
        testName - the name of test to display on the status screen
        duration - the amount of time to run fuzzing for, where null indicates unlimited time.
        outputDirectory - the directory where fuzzing results will be written
        Throws:
        IOException - if the output directory could not be prepared
    • Method Detail

      • getStatNames

        protected String getStatNames()
      • infoLog

        protected void infoLog​(String str,
                               Object... args)
      • millisToDuration

        protected String millisToDuration​(long millis)
      • displayStats

        protected void displayStats​(boolean force)
      • updateCoverageFile

        protected void updateCoverageFile()
        Updates the data in the coverage file
      • getTitle

        protected String getTitle()
      • setBlind

        public void setBlind​(boolean blind)
      • getTargetChildrenForParent

        protected int getTargetChildrenForParent​(ZestGuidance.Input parentInput)
      • completeCycle

        protected void completeCycle()
        Handles the end of fuzzing cycle (i.e., having gone through the entire queue)
      • createFreshInput

        protected ZestGuidance.Input<?> createFreshInput()
        Spawns a new input from thin air (i.e., actually random)
        Returns:
        a fresh input
      • createParameterStream

        protected InputStream createParameterStream()
        Returns an InputStream that delivers parameters to the generators. Note: The variable `currentInput` has been set to point to the input to mutate.
        Returns:
        an InputStream that delivers parameters to the generators
      • getInput

        public InputStream getInput()
                             throws GuidanceException
        Description copied from interface: Guidance
        Returns a reference to a stream of values return from the pseudo-random number generator.

        This method is guaranteed to be invoked by JQF at most once after each invocation of Guidance.hasInput() that has returned true.

        If Guidance.hasInput() returns false or has not been invoked since the last call to Guidance.getInput(), then invoking this method may throw an IllegalStateException.

        Specified by:
        getInput in interface Guidance
        Returns:
        a stream of bytes to be used by the input generator(s)
        Throws:
        GuidanceException - if there was an I/O or other error in generating the input stream
      • hasInput

        public boolean hasInput()
        Description copied from interface: Guidance
        Returns whether an input is ready for a new trial to be executed.

        This method may block until a new input stream is made ready. If this method returns false, then JQF stops fuzzing and this guidance will not be used further.

        Specified by:
        hasInput in interface Guidance
        Returns:
        whether a new trial should be executed
      • handleResult

        public void handleResult​(Result result,
                                 Throwable error)
                          throws GuidanceException
        Description copied from interface: Guidance
        Handles the end of a fuzzing trial.

        This method is guaranteed to be invoked by JQF exactly once after each invocation of Guidance.getInput(). Therefore, it is safe to open resources such as files in a call to Guidance.getInput() and only close them inside this method.

        If result is SUCCESS, then error is either null or it is an instance of a throwable that is declared by the test method in its throws clause.

        If result is INVALID, then error is either an AssumptionViolatedException, if the argument of an assume() statement was false, or it is a GuidanceException, indicating that fuzzing was interrupted during the execution of this trial (and will not continue further).

        If result is FAILURE, then error is some other throwable that was thrown by the test execution but was not listed in its throws clause. This is the only way to detect test errors. Assertion errors will typically fall into this category. So will other unlisted RuntimeExceptions such as NPE.

        Specified by:
        handleResult in interface Guidance
        Parameters:
        result - the result of the fuzzing trial
        error - the error thrown during the trial, or null
        Throws:
        GuidanceException - if there was an I/O or other error in handling the result
      • checkSavingCriteriaSatisfied

        protected List<String> checkSavingCriteriaSatisfied​(Result result)
      • computeResponsibilities

        protected org.eclipse.collections.impl.set.mutable.primitive.IntHashSet computeResponsibilities​(boolean valid)
      • saveCurrentInput

        protected void saveCurrentInput​(org.eclipse.collections.impl.set.mutable.primitive.IntHashSet responsibilities,
                                        String why)
                                 throws IOException
        Throws:
        IOException
      • generateCallBack

        public Consumer<TraceEvent> generateCallBack​(Thread thread)
        Description copied from interface: Guidance
        Returns a callback generator for a thread's event trace.

        The application under test is instrumented such that each thread generates a sequence of TraceEvents that may be handled by a separate callback method (though it may also be the same callback).

        The callback provided by this method will typically be used for collection execution information such as branch coverage, which in turn is used for constructing the next input stream.

        This method is a supplier of event consumers. It is invoked once per new application thread spawned during fuzzing.

        Specified by:
        generateCallBack in interface Guidance
        Parameters:
        thread - the thread whose events to handle
        Returns:
        a callback that handles trace events generated by that thread
      • handleEvent

        protected void handleEvent​(TraceEvent e)
        Handles a trace event generated during test execution. Not used by FastNonCollidingCoverage, which does not allocate an instance of TraceEvent at each branch probe execution.
        Parameters:
        e - the trace event to be handled
      • getTotalCoverage

        public ICoverage getTotalCoverage()
        Returns a reference to the coverage statistics.
        Returns:
        a reference to the coverage statistics
      • conditionallySynchronize

        protected void conditionallySynchronize​(boolean cond,
                                                Runnable task)
        Conditionally run a method using synchronization. This is used to handle multi-threaded fuzzing.