Interface Guidance
-
- All Known Subinterfaces:
DiffFuzzGuidance
- All Known Implementing Classes:
AFLGuidance
,DiffFuzzNoGuidance
,DiffFuzzReproGuidance
,ExecutionIndexingGuidance
,NoGuidance
,PerfFuzzGuidance
,ReproGuidance
,ReproServerGuidance
,ZestGuidance
public interface Guidance
A front-end for guided fuzzing.Before each fuzzing trial, the front-end is queried for input to be used as a source of random numbers by junit-quickcheck's generators.
During a fuzzing trial, trace events are generated, and the front-end provides callbacks to handle these trace events (e.g. to collect branch coverage).
At the end of a fuzzing trial, the front-end is notified of the result of the trial.
The standard sequence of method invocations on a Guidance instance is as follows (in pseudo-code):
while (guidance.hasInput()) { Random rng = new StreamBackedRandom(guidance.getInput()); Object[] args = generateInput(rng); try { runTest(args); // generates many trace events guidance.handleResult(SUCCESS, null); } catch (AssumptionViolatedException e) { guidance.handleResult(INVALID, e); } catch (Throwable t) { if (isExpected(e)) { guidance.handleResult(SUCCESS, e); } else { guidance.handleResult(FAILURE, e); } } }
See the implementation ofFuzzStatement
for the real loop.
-
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description static InputStream
createInputStream(Supplier<Integer> inputByteSource)
Consumer<TraceEvent>
generateCallBack(Thread thread)
Returns a callback generator for a thread's event trace.InputStream
getInput()
Returns a reference to a stream of values return from the pseudo-random number generator.void
handleResult(Result result, Throwable error)
Handles the end of a fuzzing trial.boolean
hasInput()
Returns whether an input is ready for a new trial to be executed.default void
observeGeneratedArgs(Object[] args)
Callback for observing actual arguments passed to the test method.default void
run(TestClass testClass, FrameworkMethod method, Object[] args)
Runs a test method with generated arguments as input.
-
-
-
Method Detail
-
getInput
InputStream getInput() throws IllegalStateException, GuidanceException
Returns a reference to a stream of values return from the pseudo-random number generator.This method is guaranteed to be invoked by JQF at most once after each invocation of
hasInput()
that has returnedtrue
.If
hasInput()
returnsfalse
or has not been invoked since the last call togetInput()
, then invoking this method may throw an IllegalStateException.- Returns:
- a stream of bytes to be used by the input generator(s)
- Throws:
IllegalStateException
- if the lasthasInput()
returnedfalse
GuidanceException
- if there was an I/O or other error in generating the input stream
-
hasInput
boolean hasInput()
Returns whether an input is ready for a new trial to be executed.This method may block until a new input stream is made ready. If this method returns
false
, then JQF stops fuzzing and this guidance will not be used further.- Returns:
- whether a new trial should be executed
-
observeGeneratedArgs
default void observeGeneratedArgs(Object[] args)
Callback for observing actual arguments passed to the test method.This method is invoked exactly once after each call to
getInput()
. The arguments to this callback are the structured inputs that are produced by junit-quickcheck generators, which in turn decode the bytes produced bygetInput()
.This method is useful for logging the generated args or for calculating the size of the generated args. The default implementation does nothing.
- Parameters:
args
- an array of arguments that will be passed to the test method; the size of this array is equal to the number of formal parameters to the test method
-
handleResult
void handleResult(Result result, Throwable error) throws GuidanceException
Handles the end of a fuzzing trial.This method is guaranteed to be invoked by JQF exactly once after each invocation of
getInput()
. Therefore, it is safe to open resources such as files in a call togetInput()
and only close them inside this method.If
result
isSUCCESS
, thenerror
is eithernull
or it is an instance of a throwable that is declared by the test method in itsthrows
clause.If
result
isINVALID
, thenerror
is either anAssumptionViolatedException
, if the argument of anassume()
statement wasfalse
, or it is aGuidanceException
, indicating that fuzzing was interrupted during the execution of this trial (and will not continue further).If
result
isFAILURE
, thenerror
is some other throwable that was thrown by the test execution but was not listed in itsthrows
clause. This is the only way to detect test errors. Assertion errors will typically fall into this category. So will other unlisted RuntimeExceptions such as NPE.- Parameters:
result
- the result of the fuzzing trialerror
- the error thrown during the trial, ornull
- Throws:
GuidanceException
- if there was an I/O or other error in handling the result
-
generateCallBack
Consumer<TraceEvent> generateCallBack(Thread thread)
Returns a callback generator for a thread's event trace.The application under test is instrumented such that each thread generates a sequence of
TraceEvent
s that may be handled by a separate callback method (though it may also be the same callback).The callback provided by this method will typically be used for collection execution information such as branch coverage, which in turn is used for constructing the next input stream.
This method is a supplier of event consumers. It is invoked once per new application thread spawned during fuzzing.
- Parameters:
thread
- the thread whose events to handle- Returns:
- a callback that handles trace events generated by that thread
-
createInputStream
static InputStream createInputStream(Supplier<Integer> inputByteSource)
-
run
default void run(TestClass testClass, FrameworkMethod method, Object[] args) throws Throwable
Runs a test method with generated arguments as input.By default, this method simply runs the test method using a JUnit
TrialRunner
. Guidances can override this method to customize how test execution should be performed once inputs are generated. For example, a guidance that supports non-deterministic test code may wish to execute multiple trials per generated input.- Parameters:
testClass
- the test classmethod
- the test method within the test classargs
- the arguments to the test method (i.e., the test input)- Throws:
Throwable
- any exception that may be thrown during test execution
-
-