Visitar URL original
Fuzz mode as documented in https://github.com/google/fuzztest/blob/main/doc/quickstart-cmake.md doesn't work · Issue #1022 · google/fuzztest · GitHub
Skip to content

Fuzz mode as documented in https://github.com/google/fuzztest/blob/main/doc/quickstart-cmake.md doesn't work #1022

Open

Description

@arvid-norlander

I followed https://github.com/google/fuzztest/blob/main/doc/quickstart-cmake.md exactly. When I tried with -DFUZZTEST_FUZZING_MODE=on it turns out it doesn't actually work:

Note: Google Test filter = MyTestSuite.IntegerAdditionCommutes
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from MyTestSuite
[ RUN      ] MyTestSuite.IntegerAdditionCommutes
FUZZTEST_PRNG_SEED=FEIOIwS7oGOzZwJD92Joo9IKQL1v4SFl1EB0uWiBrqU


[!] To fuzz, please build with --config=fuzztest.


/home/tmgarn/workspace/cmake_reproducer/fuzztest/./fuzztest/internal/googletest_adaptor.h:66: Failure
Expected equality of these values:
  0
  test->RunInFuzzingMode(argc_, argv_, configuration_)
    Which is: 1
Fuzzing failure.
Stack trace:
  0x559830210fb9: fuzztest::internal::GTest_TestAdaptor::TestBody()
  0x5598302a1adb: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
  0x55983028633a: testing::internal::HandleExceptionsInMethodIfSupported<>()
  0x559830264783: testing::Test::Run()
  0x559830265380: testing::TestInfo::Run()
... Google Test internal frames ...


=================================================================
=== Fuzzing stats

Elapsed time: 2.231779ms
Total runs: 0
Edges covered: 0
Total edges: 0
Corpus size: 0
Max stack used: 0

==================================

Digging into the code it appears that this depends on something called "centipede" for which there is no code in the Cmake build system, only the bazel one seems to have support

Activity

  1. formatc2013 commented on Mar 22, 2024

    @formatc2013

    I only run into this if I compile without:
    set(FUZZTEST_FUZZING_MODE ON)

    My CMakeLists.txt

    
    cmake_minimum_required(VERSION 3.19)
    project(first_fuzz_project)
    
    set(FUZZTEST_FUZZING_MODE ON)
    
    set(CMAKE_CXX_STANDARD 20)
    
    
    add_subdirectory(fuzztest)
    set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
    set(THREADS_PREFER_PTHREAD_FLAG TRUE)
    find_package(Threads REQUIRED)
    
    enable_testing()
    
    include(GoogleTest)
    
    fuzztest_setup_fuzzing_flags()
    
    add_executable(first_fuzz_test common.h first_fuzz_test.cpp)
    
    target_link_libraries(first_fuzz_test PRIVATE xyz)
    
    link_fuzztest(first_fuzz_test)
    gtest_discover_tests(first_fuzz_test)
    set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
    
    

    Fails with:

    [1/2] Linking CXX executable output/first_fuzz_test
    
    ==19646==The following global variable is not properly aligned.
    ==19646==This may happen if another global with the same name
    ==19646==resides in another non-instrumented module.
    ==19646==Or the global comes from a C file built w/o -fno-common.
    ==19646==In either case this is likely an ODR violation bug,
    ==19646==but AddressSanitizer can not provide more details.
    =================================================================
    ==19646==ERROR: AddressSanitizer: odr-violation (0x5585d812aecf):
      [1] size=35 'typeinfo name for testing::internal::DeathTestImpl' deps/googletest-src/googletest/src/gtest-all.cc
      [2] size=35 'typeinfo name for testing::internal::DeathTestImpl' deps/googletest-src/googletest/src/gtest-all.cc
    These globals were registered at these points:
    ...
        #2 0x7f06f57dc47d in call_init elf/./elf/dl-init.c:70:3
    
    ==4120==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
    SUMMARY: AddressSanitizer: odr-violation: global 'typeinfo name for testing::internal::DeathTestImpl' at 
    

    export ASAN_OPTIONS=detect_odr_violation=0
    seems to have no effect

    This helps:
    add_compile_options(-mllvm -asan-use-private-alias=1)

    but than I have to:
    export ASAN_OPTIONS=alloc_dealloc_mismatch=0

    and than back at your problem

  2. cyclicreferences commented on Jun 6, 2024

    @cyclicreferences

    @arvid-norlander did you manage to get it working with cmake?

  3. cyclicreferences commented on Jun 9, 2024

    @cyclicreferences

    I could get it working in compatibility mode with libfuzzer.
    The errors I received was because I publicly linked GTest somewhere else and the cmake from the setup also links against it

  4. arvid-norlander commented on Jun 10, 2024

    @arvid-norlander
    Author

    I managed to get it working partially, but it was really finicky and didn't work well with the rest of our cmake build system ("backend" is a terrible name in anything that has a global namespace for names, neither fuzztest nor our code should be using it!). So I did a short test, but never anything that got incorporated into our CI.

    I never got the continuous fuzzing mode working at all.

    I'm more likely to look at other solutions in the future than this project. It seems to be the typical google "dump code as FOSS, but don't make it well documented or work well and ignore all outside bugs and contributions". Don't really see what google is getting out of that to be honest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions