better documentation of Git-sourced fields when parsing

The fact we use the Git history to deduce the published and modified date fields is
non-obvious and can confuse users, because parsing our advisory content will fail if
they are divorced from the Git repo. We should improve the documentation about this.

original summary retained below

Summary

remove an advisory
toml parsing fails

Expected behavior

toml parsing works on some bag of advisories, there shouldn't have to be any constraints on them

Cause of the bug

yeah I have no idea of where this is coming from; the toml parser also fails if I just 1;1 copy the advisories to some empty directories.

ListAdvisoryValidationError
    "/nix/store/99blf5rfm6rp0sr18168xclkw1gnr47y-empty-dir-with-advisories"
    [ AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    , AdvisoryError
        [ MatchMessage
            { matchAnn = Nothing
            , matchPath = []
            , matchMessage =
                "missing mandatory key: advisory.date"
            }
        ]
        "missing mandatory key: advisory.date in <top-level>\n"
    ]

is it possible that this has to do something with .git? this is the only way I could explain how just copying it to the store would break it.

We use the git history to deduce the date and modified data. Advisory files that are not in a git repo need to include those fields explicitly.

I can't reproduce the issue as described in the summary (remove one advisory). I deduce that this all
occurred outside a git repo, which is the actual cause (and expected behaviour).

FWIW, we will soon work on an index/archive format of some kind (#170) for efficient distribution of advisories, and which
will allow them to be divorced from the git repo without losing metadata. In it's simplest form, this could be a
re-rendering of the advisories with the derived date fields present in the TOML.

Ahhh makes sense yes. Thank you!

FWIW, we will soon work on an index/archive format of some kind (#170) for efficient distribution of advisories, and which
will allow them to be divorced from the git repo without losing metadata. In it's simplest form, this could be a
re-rendering of the advisories with the derived date fields present in the TOML.

I think that's a good idea; fwiw if you provided not-only a toml parser but also pretty-printer, that should be really easy to do ;)

Cheers, I'm going to leave this issue open as a reminder to improve the documentation about this implementation detail.

thank you <3

osv seems to be appropriate

I guess as a jsonl

@frasertweedale

without any code changes, this started breaking downstream @frasertweedale https://github.com/MangoIV/cabal-audit/actions/runs/9102576510/job/25656820603 I don't know what is wrong, I can use cabal-audit as usual locally, it only fails in the github runner now. I don't know what the issues is there.

Navigation Menu

Repository navigation

better documentation of Git-sourced fields when parsing #176

Summary

Expected behavior

Cause of the bug

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

better documentation of Git-sourced fields when parsing #176

Description

Summary

Expected behavior

Cause of the bug

Activity

MangoIV commented on Mar 29, 2024

MangoIV commented on Mar 29, 2024

MangoIV commented on Mar 29, 2024

frasertweedale commented on Mar 29, 2024

frasertweedale commented on Mar 29, 2024

MangoIV commented on Mar 29, 2024

MangoIV commented on Mar 30, 2024

frasertweedale commented on Mar 30, 2024

MangoIV commented on Mar 30, 2024

MangoIV commented on Mar 31, 2024

MangoIV commented on Mar 31, 2024

MangoIV commented on May 31, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions