Object properties should not be re-sorted alphabetically

Usually JSON documents have a meaningful ordering of properties, and rarely it is alphabetical.

With this library all JSON data gets sorted alphabetically, which is undesirable.

I guess it must be due to use of std::map.

Please describe the steps to reproduce the issue.

#include <iostream>
#include "json.hpp"

using Json = nlohmann::json;

int main() {
  std::cout << ">>> " << Json{{"C",1},{"B",2},{"A",3}}.dump() << std::endl;
}

What is the expected behavior?

>>> {"C":1, "B":2, "A":3}

And what is the actual behavior instead?

{"A":3,"B":2,"C":1}

Which compiler and operating system are you using?

Compiler: clang-7 and gcc-7
Operating system: Ubuntu 18

Which version of the library did you use?

[ x ] latest release version 3.7.3

This behavior is described in the readme including ways to circumvent it.

Thanks, I had missed the Order of object keys section in README.
Great library BTW.

I just want to highlight that some unordered_map example in the README does not seem to work:

std::unordered_map<const char*, double> c_umap { {"one", 1.2}, {"two", 2.3}, {"three", 3.4} };
json j_umap(c_umap);
// {"one": 1.2, "two": 2.3, "three": 3.4}

This produces exactly the same as the default:

cout << j_umap.dump() << endl;
=> {"one":1.2,"three":3.4,"two":2.3}`.

The README does not mention using std::unordered_map as workaround, but tsl::unordered_map or nlohmann::fifo_map, see https://github.com/nlohmann/json/blob/develop/README.md#order-of-object-keys.

Sure though the commented output under std::unordered_map above is misguiding. I suggest correcting it to highlight the actual order.

To preserve insertion order in my use case, I made a minimal solution based on std::vector.
Leaving it here for those who for any reason don't want to import further non-standard dependencies, or want a starting point to build their own:

/// VecMap: a minimal map-like container that preserves insertion order
/// for use within nlohmann::basic_json<VecMap>
template<class Key, class T, class IgnoredLess=std::less<Key>,
    class Allocator = std::allocator<std::pair<const Key, T>>,
    class Container = std::vector<std::pair<const Key, T>, Allocator>>
struct VecMap : Container
{
    using key_type = Key;
    using mapped_type = T;    
    using value_type = std::pair<const Key, T>;
    using Container::Container;
    auto emplace( Key && key, T && t ) {
        for (auto it=this->begin(); it !=this->end(); ++it)
            if (it->first == key)
                return std::pair {it, false};
        this->emplace_back(key, t);
        return std::pair {--this->end(), true};
    }
    T& operator[]( Key && key ) {
        return emplace(std::move(key), T{}).first->second;
    }
};

Again, thanks for this great JSON library!

Your VecMap container is very nice and compact. Can I use this as example in the documentation?

Of course 👍

It works well in my tests.
It is an effective choice for small objects due to the low overhead of std::vector. And its algorithmic simplicity makes it a good target for compiler optimization.

    using Json = nlohmann::json;
    using FifoJson = nlohmann::basic_json<VecMap>;

    vector<pair<string,string>> vec {{"Z","1"},{"Y","2"},{"X","3"}};
    cout << Json(vec) << endl;
   // => [["Z","1"],["Y","2"],["X","3"]] - array, not object

    VecMap<string,string> vm {{"Z","1"}};
    vm["Z"] = "2";
    vm["Y"] = "3";
    vm["X"] = "4";
    cout << Json(vm) << endl;  // VecMap::mapped_type makes it look like a map ~ object
    // => {"X":"4","Y":"3","Z":"2"}

    cout << FifoJson(vm) << endl;
    // => {"Z":"2","Y":"3","X":"4"}  - Object, in insertion order

    FifoJson fj {{"Z","1"},{"Y","2"},{"X",3}};
    fj["Z"] = 4;
    fj["A"] = 5;
    cout << fj << endl;
    // => {"Z":4,"Y":"2","X":3,"A":5}

It was missing using mapped_type = T to be fully recognized. Just added it and updated the above comments...

@gatopeich

@gatopeich

I am currently trying to put this all together to add this as ordered_json to the library. While writing tests, I saw that removing elements is not yet working. This is my current approach:

std::size_t erase(const key_type& key)
{
    std::size_t result = 0;
    for (auto it = this->begin(); it != this->end();)
    {
        if (it->first == key)
        {
            ++result;
            it = Container::erase(it);
        }
        else
        {
            ++it;
        }
    }

    return result;
}

which fails due to an error in it = Container::erase(it);:

cannot be assigned because its copy assignment operator is implicitly deleted

Any ideas? Here is the current state: 4fd0d02

The problem is in the definition of object_t there is an assumption that the allocator is for a const string:

    using object_t = ObjectType<StringType,
          basic_json,
          object_comparator_t,
          AllocatorType<std::pair<const StringType,
          basic_json>>>;

And a const string may have a specialized implementation, e.g. with smaller allocation requirements, thus with incompatible allocator...

I took the default template arguments from std::map:

template<
    class Key,
    class T,
    class Compare = std::less<Key>,
    class Allocator = std::allocator<std::pair<const Key, T> >
> class map;

I'm afraid changing this would break the API. Any idea how to fix the "vector of pairs" approach? Maybe using two vectors?

I see 3 ways forward:

Optimal: find if C++11 has some way to put AllocatorType<Container::value_type> inside the declaration of object_t. Note that Container::value_type == object_t::value_type == basic_json::value_type so it is a self-referencing template... Seems beyond my template abilities, but it would clear the way to any Container class, not just std::vector.

Probably the cleanest way is by wrapping std::map:

template<class Key, class T, template<class> AllocatorType>
class MapWrapper : std:map<Key, T, std::less<Key>, AllocatorType<const Key, T>> {}

then ObjectType = MapWrapper, and object_t declaration would be much simpler:

object_t = ObjectType<StringType, basic_json, AllocatorType>;

Only I would not want to go into mayor refactoring 🙂 so...

Re-implement vector::erase() without relying on pair move operations missing due to const member. e.g. by using in-place destructor + constructor. I am working at this now.
Use std::forward_list instead of std::vector. Me not like it, but should work.

@nlohmann

Hey @nlohmann my pull request #2206 is ready and waiting...

Thanks for taking this in!

@gatopeich

great feature @gatopeich

@nlohmann

@nlohmann I think this should have been named unordered_json, in-line with what std::map, std::unordered_map, std::set, std::unordered_set do.

std::map orders its elements in a particular fashion, while std::unordered_map preserves the order in which its elements are inserted. So std::map is actually an alias for ordered_map

Similarly nhlomann::json should have been alias for ordered_json as implicitly sorts the elements and nhlomann::unordered_json should preserve the order of insertion (i.e. it does not sort). 😄

std::map orders its elements in a particular fashion, while std::unordered_map preserves the order in which its elements are inserted.

This is not true. In std::unordered_map, elements are hashed and stored in buckets.

Similarly nhlomann::json should have been alias for ordered_json as implicitly sorts the elements and nhlomann::unordered_json should preserve the order of insertion (i.e. it does not sort). 😄

See https://json.nlohmann.me/features/object_order/ on the rationale.

Navigation Menu

Object properties should not be re-sorted alphabetically #2179

Please describe the steps to reproduce the issue.

What is the expected behavior?

And what is the actual behavior instead?

Which compiler and operating system are you using?

Which version of the library did you use?

16 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Object properties should not be re-sorted alphabetically #2179

Description

Please describe the steps to reproduce the issue.

What is the expected behavior?

And what is the actual behavior instead?

Which compiler and operating system are you using?

Which version of the library did you use?

Activity

nlohmann commented on Jun 8, 2020

gatopeich commented on Jun 8, 2020

gatopeich commented on Jun 8, 2020

nlohmann commented on Jun 8, 2020

gatopeich commented on Jun 10, 2020

nlohmann commented on Jun 10, 2020

gatopeich commented on Jun 10, 2020

gatopeich commented on Jun 10, 2020

gatopeich commented on Jun 10, 2020

nlohmann commented on Jun 19, 2020

16 remaining items

gatopeich commented on Jun 21, 2020

nlohmann commented on Jun 22, 2020

gatopeich commented on Jun 22, 2020

gatopeich commented on Jul 6, 2020

gatopeich commented on Jul 17, 2020

shdxiang commented on Dec 11, 2020

pkbehera commented on Oct 27, 2022

nlohmann commented on Oct 27, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions