Package edu.berkeley.cs.jqf.fuzz.ei

Class ZestGuidance

java.lang.Object
edu.berkeley.cs.jqf.fuzz.ei.ZestGuidance
All Implemented Interfaces:
Guidance
Direct Known Subclasses:
ExecutionIndexingGuidance
public class ZestGuidance extends Object implements Guidance
A guidance that performs coverage-guided fuzzing using two coverage maps, one for all inputs and one for valid inputs only.
Author:
Rohan Padhye

  • Field Details

    • MUTATION_ZERO_PROBABILITY

      protected final double MUTATION_ZERO_PROBABILITY
      Probability that a standard mutation sets the byte to just zero instead of a random value.
      See Also:

    • random

      protected Random random
      A pseudo-random number generator for generating fresh values.

    • testName

      protected final String testName
      The name of the test for display purposes.

    • maxDurationMillis

      protected final long maxDurationMillis
      The max amount of time to run for, in milli-seconds

    • maxTrials

      protected final long maxTrials
      The max number of trials to run

    • numTrials

      protected long numTrials
      The number of trials completed.

    • numValid

      protected long numValid
      The number of valid inputs.

    • outputDirectory

      protected final File outputDirectory
      The directory where fuzzing results are produced.

    • savedCorpusDirectory

      protected File savedCorpusDirectory
      The directory where interesting inputs are saved.

    • savedFailuresDirectory

      protected File savedFailuresDirectory
      The directory where saved inputs are saved.

    • allInputsDirectory

      protected File allInputsDirectory
      The directory where all generated inputs are logged in sub-directories (if enabled).

    • savedInputs

      protected ArrayList<ZestGuidance.Input> savedInputs
      Set of saved inputs to fuzz.

    • seedInputs

      protected Deque<ZestGuidance.Input> seedInputs
      Queue of seeds to fuzz.

    • currentInput

      protected ZestGuidance.Input<?> currentInput
      Current input that's running -- valid after getInput() and before handleResult().

    • currentParentInputIdx

      protected int currentParentInputIdx
      Index of currentInput in the savedInputs -- valid after seeds are processed (OK if this is inaccurate).

    • numChildrenGeneratedForCurrentParentInput

      protected int numChildrenGeneratedForCurrentParentInput
      Number of mutated inputs generated from currentInput.

    • cyclesCompleted

      protected int cyclesCompleted
      Number of cycles completed (i.e. how many times we've reset currentParentInputIdx to 0.

    • numFavoredLastCycle

      protected int numFavoredLastCycle
      Number of favored inputs in the last cycle.

    • blind

      protected boolean blind
      Blind fuzzing -- if true then the queue is always empty.

    • validityFuzzing

      protected boolean validityFuzzing
      Validity fuzzing -- if true then save valid inputs that increase valid coverage

    • numSavedInputs

      protected int numSavedInputs
      Number of saved inputs. This is usually the same as savedInputs.size(), but we do not really save inputs in TOTALLY_RANDOM mode.

    • runCoverage

      protected ICoverage runCoverage
      Coverage statistics for a single run.

    • totalCoverage

      protected ICoverage totalCoverage
      Cumulative coverage statistics.

    • validCoverage

      protected ICoverage validCoverage
      Cumulative coverage for valid inputs.

    • maxCoverage

      protected int maxCoverage
      The maximum number of keys covered by any single input found so far.

    • responsibleInputs

      protected Map<Object,ZestGuidance.Input> responsibleInputs
      A mapping of coverage keys to inputs that are responsible for them.

    • uniqueFailures

      protected Set<String> uniqueFailures
      The set of unique failures found so far.

    • EXACT_CRASH_PATH

      protected final String EXACT_CRASH_PATH
      save crash to specific location (should be used with EXIT_ON_CRASH)

    • verbose

      protected final boolean verbose
      Whether to print log statements to stderr (debug option; manually edit).
      See Also:

    • console

      protected final Console console
      A system console, which is non-null only if STDOUT is a console.

    • startTime

      protected final Date startTime
      Time since this guidance instance was created.

    • lastRefreshTime

      protected Date lastRefreshTime
      Time at last stats refresh.

    • lastNumTrials

      protected long lastNumTrials
      Total execs at last stats refresh.

    • STATS_REFRESH_TIME_PERIOD

      protected final long STATS_REFRESH_TIME_PERIOD
      Minimum amount of time (in millis) between two stats refreshes.
      See Also:

    • logFile

      protected File logFile
      The file where log data is written.

    • statsFile

      protected File statsFile
      The file where saved plot data is written.

    • currentInputFile

      protected File currentInputFile
      The currently executing input (for debugging purposes).

    • coverageFile

      protected File coverageFile
      The file contianing the coverage information

    • LIBFUZZER_COMPAT_OUTPUT

      protected final boolean LIBFUZZER_COMPAT_OUTPUT
      Use libFuzzer like output instead of AFL like stats screen (https://llvm.org/docs/LibFuzzer.html#output)

    • QUIET_MODE

      protected final boolean QUIET_MODE
      Whether to hide fuzzing statistics

    • LOG_ALL_INPUTS

      protected final boolean LOG_ALL_INPUTS
      Whether to store all generated inputs to disk (can get slowww!)

    • LOG_CURRENT_INPUT

      protected final boolean LOG_CURRENT_INPUT
      Controls whether to update the .cur_input file with the most recently generated input (disabling improves performance). Defaults to true.

    • singleRunTimeoutMillis

      protected long singleRunTimeoutMillis
      Timeout for an individual run.

    • runStart

      protected Date runStart
      Date when last run was started.

    • branchCount

      protected long branchCount
      Number of conditional jumps since last run was started.

    • EXIT_ON_CRASH

      protected final boolean EXIT_ON_CRASH
      Whether to stop/exit once a crash is found.

    • firstThread

      protected Thread firstThread
      The first thread in the application, which usually runs the test method.

    • multiThreaded

      protected boolean multiThreaded
      Whether the application has more than one thread running coverage-instrumented code

    • SAVE_ONLY_VALID

      protected final boolean SAVE_ONLY_VALID
      Whether to save only valid inputs

    • MAX_INPUT_SIZE

      protected final int MAX_INPUT_SIZE
      Max input size to generate.

    • GENERATE_EOF_WHEN_OUT

      protected final boolean GENERATE_EOF_WHEN_OUT
      Whether to generate EOFs when we run out of bytes in the input, instead of randomly generating new bytes.

    • NUM_CHILDREN_BASELINE

      protected final int NUM_CHILDREN_BASELINE
      Baseline number of mutated children to produce from a given parent input.
      See Also:

    • NUM_CHILDREN_MULTIPLIER_FAVORED

      protected final int NUM_CHILDREN_MULTIPLIER_FAVORED
      Multiplication factor for number of children to produce for favored inputs.
      See Also:

    • MEAN_MUTATION_COUNT

      protected final double MEAN_MUTATION_COUNT
      Mean number of mutations to perform in each round.
      See Also:

    • MEAN_MUTATION_SIZE

      protected final double MEAN_MUTATION_SIZE
      Mean number of contiguous bytes to mutate in each mutation.
      See Also:

    • DISABLE_SAVE_NEW_COUNTS

      protected final boolean DISABLE_SAVE_NEW_COUNTS
      Whether to save inputs that only add new coverage bits (but no new responsibilities).

    • STEAL_RESPONSIBILITY

      protected final boolean STEAL_RESPONSIBILITY
      Whether to steal responsibility from old inputs (this increases computation cost).

  • Constructor Details

    • ZestGuidance

      public ZestGuidance(String testName, Duration duration, Long trials, File outputDirectory, Random sourceOfRandomness) throws IOException
      Creates a new Zest guidance instance with optional duration, optional trial limit, and possibly deterministic PRNG.
      Parameters:
      testName - the name of test to display on the status screen
      duration - the amount of time to run fuzzing for, where null indicates unlimited time.
      trials - the number of trials for which to run fuzzing, where null indicates unlimited trials.
      outputDirectory - the directory where fuzzing results will be written
      sourceOfRandomness - a pseudo-random number generator
      Throws:
      IOException - if the output directory could not be prepared

    • ZestGuidance

      public ZestGuidance(String testName, Duration duration, Long trials, File outputDirectory, File[] seedInputFiles, Random sourceOfRandomness) throws IOException
      Creates a new Zest guidance instance with seed input files and optional duration, optional trial limit, an possibly deterministic PRNG.
      Parameters:
      testName - the name of test to display on the status screen
      duration - the amount of time to run fuzzing for, where null indicates unlimited time.
      trials - the number of trials for which to run fuzzing, where null indicates unlimited trials.
      outputDirectory - the directory where fuzzing results will be written
      seedInputFiles - one or more input files to be used as initial inputs
      sourceOfRandomness - a pseudo-random number generator
      Throws:
      IOException - if the output directory could not be prepared

    • ZestGuidance

      public ZestGuidance(String testName, Duration duration, Long trials, File outputDirectory, File seedInputDir, Random sourceOfRandomness) throws IOException
      Creates a new Zest guidance instance with seed input directory and optional duration, optional trial limit, an possibly deterministic PRNG.
      Parameters:
      testName - the name of test to display on the status screen
      duration - the amount of time to run fuzzing for, where null indicates unlimited time.
      trials - the number of trials for which to run fuzzing, where null indicates unlimited trials.
      outputDirectory - the directory where fuzzing results will be written
      seedInputDir - the directory containing one or more input files to be used as initial inputs
      sourceOfRandomness - a pseudo-random number generator
      Throws:
      IOException - if the output directory could not be prepared

    • ZestGuidance

      public ZestGuidance(String testName, Duration duration, File outputDirectory, File seedInputDir) throws IOException
      Creates a new Zest guidance instance with seed inputs and optional duration.
      Parameters:
      testName - the name of test to display on the status screen
      duration - the amount of time to run fuzzing for, where null indicates unlimited time.
      outputDirectory - the directory where fuzzing results will be written
      seedInputDir - the directory containing one or more input files to be used as initial inputs
      Throws:
      IOException - if the output directory could not be prepared

    • ZestGuidance

      public ZestGuidance(String testName, Duration duration, File outputDirectory) throws IOException
      Creates a new Zest guidance instance with seed inputs and optional duration.
      Parameters:
      testName - the name of test to display on the status screen
      duration - the amount of time to run fuzzing for, where null indicates unlimited time.
      outputDirectory - the directory where fuzzing results will be written
      Throws:
      IOException - if the output directory could not be prepared

    • ZestGuidance

      public ZestGuidance(String testName, Duration duration, File outputDirectory, File[] seedFiles) throws IOException
      Creates a new Zest guidance instance with seed inputs and optional duration.
      Parameters:
      testName - the name of test to display on the status screen
      duration - the amount of time to run fuzzing for, where null indicates unlimited time.
      outputDirectory - the directory where fuzzing results will be written
      Throws:
      IOException - if the output directory could not be prepared

  • Method Details

    • getStatNames

      protected String getStatNames()

    • appendLineToFile

      protected void appendLineToFile(File file, String line) throws GuidanceException
      Throws:
      GuidanceException

    • observeGuidance

      public String observeGuidance()
      Description copied from interface: Guidance

      This method is only invoked during optional observability logging to provide the type of guidance used for input generation.

      Specified by:
      observeGuidance in interface Guidance

    • infoLog

      protected void infoLog(String str, Object... args)

    • millisToDuration

      protected String millisToDuration(long millis)

    • displayStats

      protected void displayStats(boolean force)

    • updateCoverageFile

      protected void updateCoverageFile()
      Updates the data in the coverage file

    • getTitle

      protected String getTitle()

    • setBlind

      public void setBlind(boolean blind)

    • getTargetChildrenForParent

      protected int getTargetChildrenForParent(ZestGuidance.Input parentInput)

    • completeCycle

      protected void completeCycle()
      Handles the end of fuzzing cycle (i.e., having gone through the entire queue)

    • createFreshInput

      protected ZestGuidance.Input<?> createFreshInput()
      Spawns a new input from thin air (i.e., actually random)
      Returns:
      a fresh input

    • createParameterStream

      protected InputStream createParameterStream()
      Returns an InputStream that delivers parameters to the generators. Note: The variable `currentInput` has been set to point to the input to mutate.
      Returns:
      an InputStream that delivers parameters to the generators

    • getInput

      public InputStream getInput() throws GuidanceException
      Description copied from interface: Guidance
      Returns a reference to a stream of values return from the pseudo-random number generator.

      This method is guaranteed to be invoked by JQF at most once after each invocation of Guidance.hasInput() that has returned true.

      If Guidance.hasInput() returns false or has not been invoked since the last call to Guidance.getInput(), then invoking this method may throw an IllegalStateException.

      Specified by:
      getInput in interface Guidance
      Returns:
      a stream of bytes to be used by the input generator(s)
      Throws:
      GuidanceException - if there was an I/O or other error in generating the input stream

    • hasInput

      public boolean hasInput()
      Description copied from interface: Guidance
      Returns whether an input is ready for a new trial to be executed.

      This method may block until a new input stream is made ready. If this method returns false, then JQF stops fuzzing and this guidance will not be used further.

      Specified by:
      hasInput in interface Guidance
      Returns:
      whether a new trial should be executed

    • handleResult

      public void handleResult(Result result, Throwable error) throws GuidanceException
      Description copied from interface: Guidance
      Handles the end of a fuzzing trial.

      This method is guaranteed to be invoked by JQF exactly once after each invocation of Guidance.getInput(). Therefore, it is safe to open resources such as files in a call to Guidance.getInput() and only close them inside this method.

      If result is SUCCESS, then error is either null or it is an instance of a throwable that is declared by the test method in its throws clause.

      If result is INVALID, then error is either an AssumptionViolatedException, if the argument of an assume() statement was false, or it is a GuidanceException, indicating that fuzzing was interrupted during the execution of this trial (and will not continue further).

      If result is FAILURE, then error is some other throwable that was thrown by the test execution but was not listed in its throws clause. This is the only way to detect test errors. Assertion errors will typically fall into this category. So will other unlisted RuntimeExceptions such as NPE.

      Specified by:
      handleResult in interface Guidance
      Parameters:
      result - the result of the fuzzing trial
      error - the error thrown during the trial, or null
      Throws:
      GuidanceException - if there was an I/O or other error in handling the result

    • checkSavingCriteriaSatisfied

      protected List<String> checkSavingCriteriaSatisfied(Result result)

    • computeResponsibilities

      protected org.eclipse.collections.impl.set.mutable.primitive.IntHashSet computeResponsibilities(boolean valid)

    • writeCurrentInputToFile

      protected void writeCurrentInputToFile(File saveFile) throws IOException
      Throws:
      IOException

    • saveCurrentInput

      protected void saveCurrentInput(org.eclipse.collections.impl.set.mutable.primitive.IntHashSet responsibilities, String why) throws IOException
      Throws:
      IOException

    • generateCallBack

      public Consumer<TraceEvent> generateCallBack(Thread thread)
      Description copied from interface: Guidance
      Returns a callback generator for a thread's event trace.

      The application under test is instrumented such that each thread generates a sequence of TraceEvents that may be handled by a separate callback method (though it may also be the same callback).

      The callback provided by this method will typically be used for collection execution information such as branch coverage, which in turn is used for constructing the next input stream.

      This method is a supplier of event consumers. It is invoked once per new application thread spawned during fuzzing.

      Specified by:
      generateCallBack in interface Guidance
      Parameters:
      thread - the thread whose events to handle
      Returns:
      a callback that handles trace events generated by that thread

    • handleEvent

      protected void handleEvent(TraceEvent e)
      Handles a trace event generated during test execution. Not used by FastNonCollidingCoverage, which does not allocate an instance of TraceEvent at each branch probe execution.
      Parameters:
      e - the trace event to be handled

    • getTotalCoverage

      public ICoverage getTotalCoverage()
      Returns a reference to the coverage statistics.
      Returns:
      a reference to the coverage statistics

    • conditionallySynchronize

      protected void conditionallySynchronize(boolean cond, Runnable task)
      Conditionally run a method using synchronization. This is used to handle multi-threaded fuzzing.