Interface Guidance
-
- All Known Subinterfaces:
DiffFuzzGuidance
- All Known Implementing Classes:
AFLGuidance,DiffFuzzNoGuidance,DiffFuzzReproGuidance,ExecutionIndexingGuidance,NoGuidance,PerfFuzzGuidance,ReproGuidance,ReproServerGuidance,ZestGuidance
public interface GuidanceA front-end for guided fuzzing.Before each fuzzing trial, the front-end is queried for input to be used as a source of random numbers by junit-quickcheck's generators.
During a fuzzing trial, trace events are generated, and the front-end provides callbacks to handle these trace events (e.g. to collect branch coverage).
At the end of a fuzzing trial, the front-end is notified of the result of the trial.
The standard sequence of method invocations on a Guidance instance is as follows (in pseudo-code):
while (guidance.hasInput()) { Random rng = new StreamBackedRandom(guidance.getInput()); Object[] args = generateInput(rng); try { runTest(args); // generates many trace events guidance.handleResult(SUCCESS, null); } catch (AssumptionViolatedException e) { guidance.handleResult(INVALID, e); } catch (Throwable t) { if (isExpected(e)) { guidance.handleResult(SUCCESS, e); } else { guidance.handleResult(FAILURE, e); } } }See the implementation ofFuzzStatementfor the real loop.
-
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description static InputStreamcreateInputStream(Supplier<Integer> inputByteSource)Consumer<TraceEvent>generateCallBack(Thread thread)Returns a callback generator for a thread's event trace.InputStreamgetInput()Returns a reference to a stream of values return from the pseudo-random number generator.voidhandleResult(Result result, Throwable error)Handles the end of a fuzzing trial.booleanhasInput()Returns whether an input is ready for a new trial to be executed.default voidobserveGeneratedArgs(Object[] args)Callback for observing actual arguments passed to the test method.default voidrun(TestClass testClass, FrameworkMethod method, Object[] args)Runs a test method with generated arguments as input.
-
-
-
Method Detail
-
getInput
InputStream getInput() throws IllegalStateException, GuidanceException
Returns a reference to a stream of values return from the pseudo-random number generator.This method is guaranteed to be invoked by JQF at most once after each invocation of
hasInput()that has returnedtrue.If
hasInput()returnsfalseor has not been invoked since the last call togetInput(), then invoking this method may throw an IllegalStateException.- Returns:
- a stream of bytes to be used by the input generator(s)
- Throws:
IllegalStateException- if the lasthasInput()returnedfalseGuidanceException- if there was an I/O or other error in generating the input stream
-
hasInput
boolean hasInput()
Returns whether an input is ready for a new trial to be executed.This method may block until a new input stream is made ready. If this method returns
false, then JQF stops fuzzing and this guidance will not be used further.- Returns:
- whether a new trial should be executed
-
observeGeneratedArgs
default void observeGeneratedArgs(Object[] args)
Callback for observing actual arguments passed to the test method.This method is invoked exactly once after each call to
getInput(). The arguments to this callback are the structured inputs that are produced by junit-quickcheck generators, which in turn decode the bytes produced bygetInput().This method is useful for logging the generated args or for calculating the size of the generated args. The default implementation does nothing.
- Parameters:
args- an array of arguments that will be passed to the test method; the size of this array is equal to the number of formal parameters to the test method
-
handleResult
void handleResult(Result result, Throwable error) throws GuidanceException
Handles the end of a fuzzing trial.This method is guaranteed to be invoked by JQF exactly once after each invocation of
getInput(). Therefore, it is safe to open resources such as files in a call togetInput()and only close them inside this method.If
resultisSUCCESS, thenerroris eithernullor it is an instance of a throwable that is declared by the test method in itsthrowsclause.If
resultisINVALID, thenerroris either anAssumptionViolatedException, if the argument of anassume()statement wasfalse, or it is aGuidanceException, indicating that fuzzing was interrupted during the execution of this trial (and will not continue further).If
resultisFAILURE, thenerroris some other throwable that was thrown by the test execution but was not listed in itsthrowsclause. This is the only way to detect test errors. Assertion errors will typically fall into this category. So will other unlisted RuntimeExceptions such as NPE.- Parameters:
result- the result of the fuzzing trialerror- the error thrown during the trial, ornull- Throws:
GuidanceException- if there was an I/O or other error in handling the result
-
generateCallBack
Consumer<TraceEvent> generateCallBack(Thread thread)
Returns a callback generator for a thread's event trace.The application under test is instrumented such that each thread generates a sequence of
TraceEvents that may be handled by a separate callback method (though it may also be the same callback).The callback provided by this method will typically be used for collection execution information such as branch coverage, which in turn is used for constructing the next input stream.
This method is a supplier of event consumers. It is invoked once per new application thread spawned during fuzzing.
- Parameters:
thread- the thread whose events to handle- Returns:
- a callback that handles trace events generated by that thread
-
createInputStream
static InputStream createInputStream(Supplier<Integer> inputByteSource)
-
run
default void run(TestClass testClass, FrameworkMethod method, Object[] args) throws Throwable
Runs a test method with generated arguments as input.By default, this method simply runs the test method using a JUnit
TrialRunner. Guidances can override this method to customize how test execution should be performed once inputs are generated. For example, a guidance that supports non-deterministic test code may wish to execute multiple trials per generated input.- Parameters:
testClass- the test classmethod- the test method within the test classargs- the arguments to the test method (i.e., the test input)- Throws:
Throwable- any exception that may be thrown during test execution
-
-