Benchmark image added.

7 files changed, 314 insertions, 113 deletions

diff --git a/README.md b/README.md
index 30eb2096..0cc1eebf 100644
--- a/README.md
+++ b/README.md
@@ -138,20 +138,22 @@ After erasing elements found:
 

 Highlights

 ----------

-- **User friendly** - Incredible easy to use and deploy. The ***using_***-declaration instantiates the container type to use. You may pass *optional* arguments for customization of value- *comparison*, *destruction*, *cloning*, *convertion types*, and more. Most methods have similar named corresponding methods in STL.

-- **Extreme performance** - The associative containers **cmap** and **cset** are more than ***4 times faster than c++ STL equivalents***, *std::unordered_map* and *std::unordered_set*! **csmap** and **csset** are more than 20% faster than *std::unordered_map/set* Also **cdeq** is significantly faster than *std::deque* in most cases, however implementations vary between different c++ compilers. See *Performance*.

-- **Full type safety** - No error prone casting of container types and elements back and forth from your containers. Less obscure bugs in your code. The compiler will let you know when retrieving or passing wrong container or element types to the methods.

-- **Uniform API** - Methods to ***construct***, ***initialize***, ***iterate*** and ***destruct*** are intuitive and uniform across the various containers, as can be seen from the example above.

-- **Small footprint** - Both source code and generated executables are small. The executable from the above example with six different containers is *18 kb in size*, when compiled with TinyC.

-- **Dual mode compilation** - Can be used a simple header-only library with static methods (default), or as a traditional library by defining symbol STC_HEADER in your project. See below for instructions.

+- **User friendly** - Incredible easy to use and deploy. Just include the header and you are good to go.  The API and functionality is very close to c++ STL, and is fully listed in the docs. The ***using***-declaration instantiates the container type to use. You may pass *optional* arguments to it for customization of value- *comparison*, *destruction*, *cloning*, *convertion types*, and more.

+- **Amazing performance** - All the containers are either about equal or faster than c++ STL containers. **cmap** and **cset** are in general around ***3 times faster*** than the c++ STL equivalents with the included generic hash key, and **csmap** and **csset** have 20% faster lookup time than *std::map / std::set* See *Performance* below.

+- **Fully memory managed** - As stated above, all containers will destruct keys, values via destructor passed as macro parameters to the ***using***-declaration. Also smart pointers are supported to be stored, see **csptr**.

+- **Full type safety** - Avoids error-prone casting of container types and elements back and forth from your containers.

+- **Uniform API** - Methods to ***construct***, ***initialize***, ***iterate*** and ***destruct*** have a uniform and intuitive usage across the various containers.

+- **Small footprint** - Small source code and generated executables. The executable from the above example with six different containers is *26 kb in size* compiled with TinyC.

+- **Dual mode compilation** - By default it is a simple header-only library with inline and static methods only, but you can easily switch to create a traditional library with shared symbols, without changing existing source files. See below how to.

 

 Installation

 ------------

 

-Because it is headers-only, files can simply be included in your program. The methods will be static by default (some inlined). You may add the project folder to CPATH environment variable, to let gcc, clang, or tinyc locate the headers.

+Because it is headers-only, headers can simply be included in your program. The methods are static by default (some inlined). You may add the project folder to CPATH environment variable, to let GCC, Clang, and TinyC locate the headers.

 

-If containers are extensively used accross several translation units with common instantiated container types, it is recommended to build as a "library", to minimize executable size. To enable this mode, specify **-DSTC_HEADER** as compiler option, and place all the instantiations of containers used in a single C source file, e.g.:

+If containers are used accross several translation units with common instantiated container types, it is recommended to build as a "library" to minimize the executable size. To enable this mode, specify **-DSTC_HEADER** as compiler option in your build environment, and place all the instantiations of containers used in a single C source file, e.g.:

 ```c

+// stc_libs.c

 #define STC_IMPLEMENTATION

 #include <stc/cstr.h>

 #include <stc/cmap.h>

@@ -163,65 +165,23 @@ using_cmap(ii, int, int);
 using_cset(ix, int64_t);

 using_cvec(i, int);

 using_clist(pt, struct Point);

-...

 ```

 Performance

 -----------

 

-All containers have templated intrusive elements. The unordered map and set are among the most performance critical containers. **cmap** and **cset** are among the very fastest unordered map implementations available, also considering highly optimized c++ implementations. Below are some benchmarks for this.

+All containers have templated intrusive elements. The unordered map and set are among the most performance critical containers. **cmap** and **cset** are among the very fastest unordered map implementations available, also considering highly optimized c++ implementations.

 

-Compiled with clang -O3 -x c++, v10.0 on windows, Ryzen 7 2700X CPU. Similar results with VC and g++.

+Compiled with Win-Clang++ v11, Mingw64 g++ 9.20, VC19, Linux-clang v10. CPU: Ryzen 7 2700X CPU @4Ghz. 

+The black bars indicates performance variation between the various compilers.

 

-- **CMAP** = stc/*cmap*

-- KMAP = klib/*khash*

-- UMAP = *std::unordered_map*

-- SMAP = *spp::sparse_hash_map*

-- BMAP = *ska::bytell_hash_map*

-- FMAP = *ska::flat_hash_map*

-- RMAP = *robin_hood::unordered_map*

-- HMAP = *tsl::hopscotch_map*

-```

-Random keys are in range [0, 2^24), seed = 1600720716:

-

-Unordered maps: 30000000 repeats of Insert random key + try to remove a random key:

-CMAP: time:  4.35, sum: 450000015000000, size: 8153497, erased 10922466

-KMAP: time:  3.37, sum: 450000015000000, size: 8153497, erased 10922466

-UMAP: time: 11.19, sum: 450000015000000, size: 8153497, erased 10922466

-SMAP: time: 14.21, sum: 450000015000000, size: 8153497, erased 10922466

-BMAP: time:  5.00, sum: 450000015000000, size: 8153497, erased 10922466

-FMAP: time:  4.68, sum: 450000015000000, size: 8153497, erased 10922466

-RMAP: time:  3.16, sum: 450000015000000, size: 8153497, erased 10922466

-HMAP: time:  4.49, sum: 450000015000000, size: 8153497, erased 10922466

-

-Unordered maps: Insert 30000000 index keys, then remove them in same order:

-CMAP: time:  3.27, erased 30000000

-KMAP: time:  2.86, erased 30000000

-UMAP: time: 15.80, erased 30000000

-SMAP: time: 22.60, erased 30000000

-BMAP: time:  7.93, erased 30000000

-FMAP: time:  5.91, erased 30000000

-RMAP: time:  3.22, erased 30000000

-HMAP: time:  5.91, erased 30000000

-

-Unordered maps: Insert 30000000 random keys, then remove them in same order:

-CMAP: time:  3.16, erased 13971002

-KMAP: time:  4.05, erased 13971002

-UMAP: time: 10.77, erased 13971002

-SMAP: time: 13.77, erased 13971002

-BMAP: time:  5.27, erased 13971002

-FMAP: time:  4.41, erased 13971002

-RMAP: time:  3.21, erased 13971002

-HMAP: time:  4.50, erased 13971002

-

-Unordered maps: Iterate 30000000 random keys:

-CMAP: time:  0.23, size: 13971002, sum 1344701724191145

-UMAP: time:  3.32, size: 13971002, sum 1344701724191145

-SMAP: time:  0.33, size: 13971002, sum 1344701724191145

-BMAP: time:  0.60, size: 13971002, sum 1344701724191145

-FMAP: time:  0.56, size: 13971002, sum 1344701724191145

-HMAP: time:  0.51, size: 13971002, sum 1344701724191145

-```

-From these tests *cmap*, *robin_hood* and *khash* are almost equally fast. std::unordered_map is very slow in comparison.

+![Benchmark](benchmarks/pics/benchmark.png)

+

+This shows that most of the STC containers performs fairly equal to the std counterparts, which is

+a great achievement as c++ containers have been optimized for decades. Still, **cmap** is almost 3 times

+faster on insert and erase, and has orders of magnitude faster iteration and destruction. Also notable

+is **csmap**, which has significantly faster lookup than *std::map*'s typical red-black tree

+implementation. **csmap** uses an AA-tree (Arne Andersson, 1993), which tends to create a flatter

+structure (more balanced) than red-black trees.

 

 Memory efficiency

 -----------------

diff --git a/benchmarks/cdeq_benchmark.cpp b/benchmarks/cdeq_benchmark.cpp
index 2f49e1b6..ff1c9361 100644
--- a/benchmarks/cdeq_benchmark.cpp
+++ b/benchmarks/cdeq_benchmark.cpp
@@ -8,6 +8,7 @@
 #endif

 

 enum {INSERT, ERASE, FIND, ITER, DESTRUCT, N_TESTS};

+const char* operations[] = {"insert", "erase", "find", "iter", "destruct"};

 typedef struct { time_t t1, t2; uint64_t sum; float fac; } Range;

 typedef struct { const char* name; Range test[N_TESTS]; } Sample;

 enum {SAMPLES = 3, N = 100000000};

@@ -21,7 +22,7 @@ using_cdeq(x, size_t);
 #ifdef __cplusplus

 Sample test_std_deque() {

     typedef std::deque<size_t> container;

-    Sample s = {"std-deque"};

+    Sample s = {"std,deque"};

     {

         s.test[INSERT].t1 = clock();

         container con;

@@ -50,16 +51,18 @@ Sample test_std_deque() {
      s.test[DESTRUCT].sum = 0;

      return s;

 }

+#else

+Sample test_std_deque() { Sample s = {"std-deque"}; return s;}

 #endif

 

 

 Sample test_stc_deque() {

     typedef cdeq_x container;

-    Sample s = {"stc-deque"};

+    Sample s = {"STC,deque"};

     {

         s.test[INSERT].t1 = clock();

         container con = cdeq_x_init();

-        cdeq_x_reserve(&con, N);

+        //cdeq_x_reserve(&con, N);

         stc64_srandom(seed);

         c_forrange (N/2) cdeq_x_push_front(&con, stc64_random() & mask1);

         c_forrange (N/2) { cdeq_x_push_back(&con, stc64_random() & mask1); cdeq_x_pop_front(&con); }

@@ -88,11 +91,10 @@ Sample test_stc_deque() {
      return s;

 }

 

-int main()

+int main(int argc, char* argv[])

 {

     Sample std_s[SAMPLES + 1], stc_s[SAMPLES + 1];

     c_forrange (i, int, SAMPLES) {

-        printf("deque benchmark sample %d\n", i);

         std_s[i] = test_std_deque();

         stc_s[i] = test_stc_deque();

         if (i > 0) c_forrange (j, int, N_TESTS) {

@@ -103,11 +105,9 @@ int main()
     }

     float std_sum = 0, stc_sum = 0;

     c_forrange (j, N_TESTS) { std_sum += secs(std_s[0].test[j]); stc_sum += secs(stc_s[0].test[j]); }

-    printf("Test-name, Insert, Erase, Find, Iter, Destruct, Total, Ratio\n");

-

-    printf("%s", std_s[0].name); c_forrange (j, N_TESTS) printf(", %.3f", secs(std_s[0].test[j]));

-    printf(", %.3f, 1.000\n", std_sum);

-

-    printf("%s", stc_s[0].name); c_forrange (j, N_TESTS) printf(", %.3f", secs(stc_s[0].test[j]));

-    printf(", %.3f, %.3f\n", stc_sum, std_sum/stc_sum);

+    if (argv[1][0] == '1') printf("compiler,library,container,count,operation,time,ratio\n");

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, operations[j], secs(std_s[0].test[j]), 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, "total", std_sum, 1.0f);

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, operations[j], secs(stc_s[0].test[j]), secs(std_s[0].test[j]) ? secs(stc_s[0].test[j])/secs(std_s[0].test[j]) : 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, "total", stc_sum, stc_sum/std_sum);

 }

diff --git a/benchmarks/clist_benchmark.cpp b/benchmarks/clist_benchmark.cpp
new file mode 100644
index 00000000..f8cadae3
--- /dev/null
+++ b/benchmarks/clist_benchmark.cpp
@@ -0,0 +1,110 @@
+#include <stdio.h>

+#include <time.h>

+#include <stc/crandom.h>

+#include <stc/clist.h>

+

+#ifdef __cplusplus

+#include <forward_list>

+#endif

+

+enum {INSERT, ERASE, FIND, ITER, DESTRUCT, N_TESTS};

+const char* operations[] = {"insert", "erase", "find", "iter", "destruct"};

+typedef struct { time_t t1, t2; uint64_t sum; float fac; } Range;

+typedef struct { const char* name; Range test[N_TESTS]; } Sample;

+enum {SAMPLES = 3, N = 50000000};

+

+uint64_t seed = 1, mask1 = 0xfffffff;

+

+static float secs(Range s) { return (float)(s.t2 - s.t1) / CLOCKS_PER_SEC; }

+

+using_clist(x, size_t);

+

+#ifdef __cplusplus

+Sample test_std_forward_list() {

+    typedef std::forward_list<size_t> container;

+    Sample s = {"std,forward_list"};

+    {

+        s.test[INSERT].t1 = clock();

+        container con;

+        stc64_srandom(seed);

+        c_forrange (N/2) con.push_front(stc64_random() & mask1);

+        c_forrange (N/2) con.push_front(stc64_random() & mask1);

+        s.test[INSERT].t2 = clock();

+        s.test[INSERT].sum = 0;

+        s.test[ERASE].t1 = clock();

+        c_forrange (N) con.pop_front();

+        s.test[ERASE].t2 = clock();

+        s.test[ERASE].sum = 0;

+     }{

+        container con;

+        stc64_srandom(seed);

+        c_forrange (N) con.push_front(stc64_random() & mask1);

+        s.test[ITER].t1 = clock();

+        size_t sum = 0;

+        for (auto i: con) sum += i;

+        s.test[ITER].t2 = clock();

+        s.test[ITER].sum = sum;

+        s.test[DESTRUCT].t1 = clock();

+     }

+     s.test[DESTRUCT].t2 = clock();

+     s.test[DESTRUCT].sum = 0;

+     return s;

+}

+#else

+Sample test_std_forward_list() { Sample s = {"std-forward_list"}; return s;}

+#endif

+

+

+Sample test_stc_forward_list() {

+    typedef clist_x container;

+    Sample s = {"STC,forward_list"};

+    {

+        s.test[INSERT].t1 = clock();

+        container con = clist_x_init();

+        stc64_srandom(seed);

+        c_forrange (N/2) clist_x_push_front(&con, stc64_random() & mask1);

+        c_forrange (N/2) clist_x_push_back(&con, stc64_random() & mask1);

+        s.test[INSERT].t2 = clock();

+        s.test[INSERT].sum = 0;

+        s.test[ERASE].t1 = clock();

+        c_forrange (N) clist_x_pop_front(&con);

+        s.test[ERASE].t2 = clock();

+        s.test[ERASE].sum = 0;

+        clist_x_del(&con);

+     }{

+        stc64_srandom(seed);

+        container con = clist_x_init();

+        c_forrange (N) clist_x_push_front(&con, stc64_random() & mask1);

+        s.test[ITER].t1 = clock();

+        size_t sum = 0;

+        c_foreach (i, clist_x, con) sum += *i.ref;

+        s.test[ITER].t2 = clock();

+        s.test[ITER].sum = sum;

+        s.test[DESTRUCT].t1 = clock();

+        clist_x_del(&con);

+     }

+     s.test[DESTRUCT].t2 = clock();

+     s.test[DESTRUCT].sum = 0;

+     return s;

+}

+

+int main(int argc, char* argv[])

+{

+    Sample std_s[SAMPLES + 1], stc_s[SAMPLES + 1];

+    c_forrange (i, int, SAMPLES) {

+        std_s[i] = test_std_forward_list();

+        stc_s[i] = test_stc_forward_list();

+        if (i > 0) c_forrange (j, int, N_TESTS) {

+            if (secs(std_s[i].test[j]) < secs(std_s[0].test[j])) std_s[0].test[j] = std_s[i].test[j];

+            if (secs(stc_s[i].test[j]) < secs(stc_s[0].test[j])) stc_s[0].test[j] = stc_s[i].test[j];

+            if (stc_s[i].test[j].sum != stc_s[0].test[j].sum) printf("Error in sum: test %d, sample %d\n", i, j);

+        }

+    }

+    float std_sum = 0, stc_sum = 0;

+    c_forrange (j, N_TESTS) { std_sum += secs(std_s[0].test[j]); stc_sum += secs(stc_s[0].test[j]); }

+    if (argv[1][0] == '1') printf("compiler,library,container,count,operation,time,ratio\n");

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, operations[j], secs(std_s[0].test[j]), 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, "total", std_sum, 1.0f);

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, operations[j], secs(stc_s[0].test[j]), secs(std_s[0].test[j]) ? secs(stc_s[0].test[j])/secs(std_s[0].test[j]) : 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, "total", stc_sum, stc_sum/std_sum);

+}
\ No newline at end of file
diff --git a/benchmarks/cmap_benchmark.cpp b/benchmarks/cmap_benchmark.cpp
index c81561e3..fe1cd2e9 100644
--- a/benchmarks/cmap_benchmark.cpp
+++ b/benchmarks/cmap_benchmark.cpp
@@ -8,29 +8,26 @@
 #endif

 

 enum {INSERT, ERASE, FIND, ITER, DESTRUCT, N_TESTS};

+const char* operations[] = {"insert", "erase", "find", "iter", "destruct"};

 typedef struct { time_t t1, t2; uint64_t sum; float fac; } Range;

 typedef struct { const char* name; Range test[N_TESTS]; } Sample;

-enum {SAMPLES = 3, N = 10000000};

-

+enum {SAMPLES = 2, N = 8000000};

 uint64_t seed = 1, mask1 = 0xffffffff;

 

 static float secs(Range s) { return (float)(s.t2 - s.t1) / CLOCKS_PER_SEC; }

 

-static inline uint32_t hash64(const void* data, size_t len) {

-    uint64_t x = *(const uint64_t *)data * 11400714819323198485ull;

-    return x ^ (x >> 24);

-}

 using_cmap(x, size_t, size_t, c_default_equals, c_default_hash32);

 

 #ifdef __cplusplus

-Sample test_std_map() {

+Sample test_std_unordered_map() {

     typedef std::unordered_map<size_t, size_t> container;

-    Sample s = {"std-unordered_map"};

+    Sample s = {"std,unordered_map"};

     {

         stc64_srandom(seed);

         s.test[INSERT].t1 = clock();

         container con;

-        c_forrange (i, N) con.emplace(stc64_random() & mask1, i);

+        c_forrange (i, N/2) con.emplace(stc64_random() & mask1, i);

+        c_forrange (i, N/2) con.emplace(i, i);

         s.test[INSERT].t2 = clock();

         s.test[INSERT].sum = con.size();

         stc64_srandom(seed);

@@ -41,11 +38,13 @@ Sample test_std_map() {
      }{

         container con;

         stc64_srandom(seed);

-        c_forrange (i, N) con.emplace(stc64_random() & mask1, i);

+        c_forrange (i, N/2) con.emplace(stc64_random() & mask1, i);

+        c_forrange (i, N/2) con.emplace(i, i);

         stc64_srandom(seed);

         s.test[FIND].t1 = clock();

         size_t sum = 0;

-        c_forrange (N) sum += con.find(stc64_random() & mask1)->second;

+        container::iterator it;

+        c_forrange (N) if ((it = con.find(stc64_random() & mask1)) != con.end()) sum += it->second;

         s.test[FIND].t2 = clock();

         s.test[FIND].sum = sum;

         s.test[ITER].t1 = clock();

@@ -59,17 +58,20 @@ Sample test_std_map() {
      s.test[DESTRUCT].sum = 0;

      return s;

 }

+#else

+Sample test_std_unordered_map() { Sample s = {"std-unordered_map"}; return s;}

 #endif

 

 

-Sample test_stc_map() {

+Sample test_stc_unordered_map() {

     typedef cmap_x container;

-    Sample s = {"stc-unordered_map"};

+    Sample s = {"STC,unordered_map"};

     {

         stc64_srandom(seed);

         s.test[INSERT].t1 = clock();

         container con = cmap_x_init();

-        c_forrange (i, N) cmap_x_emplace(&con, stc64_random() & mask1, i);

+        c_forrange (i, N/2) cmap_x_emplace(&con, stc64_random() & mask1, i);

+        c_forrange (i, N/2) cmap_x_emplace(&con, i, i);

         s.test[INSERT].t2 = clock();

         s.test[INSERT].sum = cmap_x_size(con);

         stc64_srandom(seed);

@@ -81,11 +83,13 @@ Sample test_stc_map() {
      }{

         container con = cmap_x_init();

         stc64_srandom(seed);

-        c_forrange (i, N) cmap_x_emplace(&con, stc64_random() & mask1, i);

+        c_forrange (i, N/2) cmap_x_emplace(&con, stc64_random() & mask1, i);

+        c_forrange (i, N/2) cmap_x_emplace(&con, i, i);

         stc64_srandom(seed);

         s.test[FIND].t1 = clock();

         size_t sum = 0;

-        c_forrange (N) sum += cmap_x_find(&con, stc64_random() & mask1).ref->second;

+        cmap_x_iter_t it;

+        c_forrange (N) if ((it = cmap_x_find(&con, stc64_random() & mask1)).ref) sum += it.ref->second;

         s.test[FIND].t2 = clock();

         s.test[FIND].sum = sum;

         s.test[ITER].t1 = clock();

@@ -101,13 +105,12 @@ Sample test_stc_map() {
      return s;

 }

 

-int main()

+int main(int argc, char* argv[])

 {

     Sample std_s[SAMPLES + 1], stc_s[SAMPLES + 1];

     c_forrange (i, int, SAMPLES) {

-        printf("unordered_map benchmark sample %d\n", i);

-        std_s[i] = test_std_map();

-        stc_s[i] = test_stc_map();

+        std_s[i] = test_std_unordered_map();

+        stc_s[i] = test_stc_unordered_map();

         if (i > 0) c_forrange (j, int, N_TESTS) {

             if (secs(std_s[i].test[j]) < secs(std_s[0].test[j])) std_s[0].test[j] = std_s[i].test[j];

             if (secs(stc_s[i].test[j]) < secs(stc_s[0].test[j])) stc_s[0].test[j] = stc_s[i].test[j];

@@ -116,11 +119,9 @@ int main()
     }

     float std_sum = 0, stc_sum = 0;

     c_forrange (j, N_TESTS) { std_sum += secs(std_s[0].test[j]); stc_sum += secs(stc_s[0].test[j]); }

-    printf("Test-name, Insert, Erase, Find, Iter, Destruct, Total, Ratio\n");

-

-    printf("%s", std_s[0].name); c_forrange (j, N_TESTS) printf(", %.3f", secs(std_s[0].test[j]));

-    printf(", %.3f, 1.000\n", std_sum);

-

-    printf("%s", stc_s[0].name); c_forrange (j, N_TESTS) printf(", %.3f", secs(stc_s[0].test[j]));

-    printf(", %.3f, %.3f\n", stc_sum, std_sum/stc_sum);

-}

+    if (argv[1][0] == '1') printf("compiler,library,container,count,operation,time,ratio\n");

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, operations[j], secs(std_s[0].test[j]), 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, "total", std_sum, 1.0f);

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, operations[j], secs(stc_s[0].test[j]), secs(std_s[0].test[j]) ? secs(stc_s[0].test[j])/secs(std_s[0].test[j]) : 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, "total", stc_sum, stc_sum/std_sum);

+}
\ No newline at end of file
diff --git a/benchmarks/csmap_benchmark.cpp b/benchmarks/csmap_benchmark.cpp
new file mode 100644
index 00000000..d0214fd2
--- /dev/null
+++ b/benchmarks/csmap_benchmark.cpp
@@ -0,0 +1,129 @@
+#include <stdio.h>

+#include <time.h>

+#include <stc/crandom.h>

+#include <stc/csmap.h>

+

+#ifdef __cplusplus

+#include <map>

+#endif

+

+enum {INSERT, ERASE, FIND, ITER, DESTRUCT, N_TESTS};

+const char* operations[] = {"insert", "erase", "find", "iter", "destruct"};

+typedef struct { time_t t1, t2; uint64_t sum; float fac; } Range;

+typedef struct { const char* name; Range test[N_TESTS]; } Sample;

+enum {SAMPLES = 2, N = 4000000};

+

+uint64_t seed = 1, mask1 = 0xfffffff;

+

+static float secs(Range s) { return (float)(s.t2 - s.t1) / CLOCKS_PER_SEC; }

+

+using_csmap(x, size_t, size_t);

+

+#ifdef __cplusplus

+Sample test_std_map() {

+    typedef std::map<size_t, size_t> container;

+    Sample s = {"std,map"};

+    {

+        stc64_srandom(seed);

+        s.test[INSERT].t1 = clock();

+        container con;

+        c_forrange (i, N/2) con.emplace(stc64_random() & mask1, i);

+        c_forrange (i, N/2) con.emplace(i, i);

+        s.test[INSERT].t2 = clock();

+        s.test[INSERT].sum = con.size();

+        stc64_srandom(seed);

+        s.test[ERASE].t1 = clock();

+        c_forrange (N) con.erase(stc64_random() & mask1);

+        s.test[ERASE].t2 = clock();

+        s.test[ERASE].sum = con.size();

+     }{

+        container con;

+        stc64_srandom(seed);

+        c_forrange (i, N/2) con.emplace(stc64_random() & mask1, i);

+        c_forrange (i, N/2) con.emplace(i, i);

+        stc64_srandom(seed);

+        s.test[FIND].t1 = clock();

+        size_t sum = 0;

+        container::iterator it;

+        c_forrange (N) if ((it = con.find(stc64_random() & mask1)) != con.end()) sum += it->second;

+        s.test[FIND].t2 = clock();

+        s.test[FIND].sum = sum;

+        s.test[ITER].t1 = clock();

+        sum = 0;

+        for (auto i: con) sum += i.second;

+        s.test[ITER].t2 = clock();

+        s.test[ITER].sum = sum;

+        s.test[DESTRUCT].t1 = clock();

+     }

+     s.test[DESTRUCT].t2 = clock();

+     s.test[DESTRUCT].sum = 0;

+     return s;

+}

+#else

+Sample test_std_map() { Sample s = {"std-map"}; return s;}

+#endif

+

+

+

+Sample test_stc_map() {

+    typedef csmap_x container;

+    Sample s = {"STC,map"};

+    {

+        stc64_srandom(seed);

+        s.test[INSERT].t1 = clock();

+        container con = csmap_x_init();

+        c_forrange (i, N/2) csmap_x_emplace(&con, stc64_random() & mask1, i);

+        c_forrange (i, N/2) csmap_x_emplace(&con, i, i);

+        s.test[INSERT].t2 = clock();

+        s.test[INSERT].sum = csmap_x_size(con);

+        stc64_srandom(seed);

+        s.test[ERASE].t1 = clock();

+        c_forrange (N) csmap_x_erase(&con, stc64_random() & mask1);

+        s.test[ERASE].t2 = clock();

+        s.test[ERASE].sum = csmap_x_size(con);

+        csmap_x_del(&con);

+     }{

+        container con = csmap_x_init();

+        stc64_srandom(seed);

+        c_forrange (i, N/2) csmap_x_emplace(&con, stc64_random() & mask1, i);

+        c_forrange (i, N/2) csmap_x_emplace(&con, i, i);

+        stc64_srandom(seed);

+        s.test[FIND].t1 = clock();

+        size_t sum = 0;

+        csmap_x_iter_t it;

+        c_forrange (N) if ((it = csmap_x_find(&con, stc64_random() & mask1)).ref) sum += it.ref->second;

+        s.test[FIND].t2 = clock();

+        s.test[FIND].sum = sum;

+        s.test[ITER].t1 = clock();

+        sum = 0;

+        c_foreach (i, csmap_x, con) sum += i.ref->second;

+        s.test[ITER].t2 = clock();

+        s.test[ITER].sum = sum;

+        s.test[DESTRUCT].t1 = clock();

+        csmap_x_del(&con);

+     }

+     s.test[DESTRUCT].t2 = clock();

+     s.test[DESTRUCT].sum = 0;

+     return s;

+}

+

+int main(int argc, char* argv[])

+{

+    Sample std_s[SAMPLES + 1], stc_s[SAMPLES + 1];

+    c_forrange (i, int, SAMPLES) {

+        std_s[i] = test_std_map();

+        stc_s[i] = test_stc_map();

+        if (i > 0) c_forrange (j, int, N_TESTS) {

+            if (secs(std_s[i].test[j]) < secs(std_s[0].test[j])) std_s[0].test[j] = std_s[i].test[j];

+            if (secs(stc_s[i].test[j]) < secs(stc_s[0].test[j])) stc_s[0].test[j] = stc_s[i].test[j];

+            if (stc_s[i].test[j].sum != stc_s[0].test[j].sum) printf("Error in sum: test %d, sample %d\n", i, j);

+        }

+    }

+    float std_sum = 0, stc_sum = 0;

+    c_forrange (j, N_TESTS) { std_sum += secs(std_s[0].test[j]); stc_sum += secs(stc_s[0].test[j]); }

+    if (argv[1][0] == '1') printf("compiler,library,container,count,operation,time,ratio\n");

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, operations[j], secs(std_s[0].test[j]), 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, "total", std_sum, 1.0f);

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, operations[j], secs(stc_s[0].test[j]), secs(std_s[0].test[j]) ? secs(stc_s[0].test[j])/secs(std_s[0].test[j]) : 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, "total", stc_sum, stc_sum/std_sum);

+}

diff --git a/benchmarks/cvec_benchmark.cpp b/benchmarks/cvec_benchmark.cpp
index 24931273..e7a4c869 100644
--- a/benchmarks/cvec_benchmark.cpp
+++ b/benchmarks/cvec_benchmark.cpp
@@ -8,9 +8,10 @@
 #endif

 

 enum {INSERT, ERASE, FIND, ITER, DESTRUCT, N_TESTS};

+const char* operations[] = {"insert", "erase", "find", "iter", "destruct"};

 typedef struct { time_t t1, t2; uint64_t sum; float fac; } Range;

 typedef struct { const char* name; Range test[N_TESTS]; } Sample;

-enum {SAMPLES = 3, N = 100000000};

+enum {SAMPLES = 3, N = 150000000};

 

 uint64_t seed = 1, mask1 = 0xfffffff;

 

@@ -21,7 +22,7 @@ using_cvec(x, size_t);
 #ifdef __cplusplus

 Sample test_std_vector() {

     typedef std::vector<size_t> container;

-    Sample s = {"std-vector"};

+    Sample s = {"std,vector"};

     {

         s.test[INSERT].t1 = clock();

         container con;

@@ -48,12 +49,15 @@ Sample test_std_vector() {
      s.test[DESTRUCT].sum = 0;

      return s;

 }

+#else

+Sample test_std_vector() { Sample s = {"std-vector"}; return s;}

 #endif

 

 

+

 Sample test_stc_vector() {

     typedef cvec_x container;

-    Sample s = {"stc-vector"};

+    Sample s = {"STC,vector"};

     {

         s.test[INSERT].t1 = clock();

         container con = cvec_x_init();

@@ -83,11 +87,10 @@ Sample test_stc_vector() {
      return s;

 }

 

-int main()

+int main(int argc, char* argv[])

 {

     Sample std_s[SAMPLES + 1] = {0}, stc_s[SAMPLES + 1] = {0};

     c_forrange (i, int, SAMPLES) {

-        printf("vector benchmark sample %d\n", i);

         std_s[i] = test_std_vector();

         stc_s[i] = test_stc_vector();

         if (i > 0) c_forrange (j, int, N_TESTS) {

@@ -98,11 +101,9 @@ int main()
     }

     float std_sum = 0, stc_sum = 0;

     c_forrange (j, N_TESTS) { std_sum += secs(std_s[0].test[j]); stc_sum += secs(stc_s[0].test[j]); }

-    printf("Test-name, Insert, Erase, Find, Iter, Destruct, Total, Ratio\n");

-

-    printf("%s", std_s[0].name); c_forrange (j, N_TESTS) printf(", %.3f", secs(std_s[0].test[j]));

-    printf(", %.3f, 1.000\n", std_sum);

-

-    printf("%s", stc_s[0].name); c_forrange (j, N_TESTS) printf(", %.3f", secs(stc_s[0].test[j]));

-    printf(", %.3f, %.3f\n", stc_sum, std_sum/stc_sum);

+    if (argv[1][0] == '1') printf("compiler,library,container,count,operation,time,ratio\n");

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, operations[j], secs(std_s[0].test[j]), 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], std_s[0].name, N, "total", std_sum, 1.0f);

+    c_forrange (j, N_TESTS) printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, operations[j], secs(stc_s[0].test[j]), secs(std_s[0].test[j]) ? secs(stc_s[0].test[j])/secs(std_s[0].test[j]) : 1.0f);

+                            printf("%s,%s,%d,%s,%.3f,%.3f\n", argv[2], stc_s[0].name, N, "total", stc_sum, stc_sum/std_sum);

 }

diff --git a/benchmarks/pics/benchmark.png b/benchmarks/pics/benchmark.png
new file mode 100644
index 00000000..9d462e3f
--- /dev/null
+++ b/benchmarks/pics/benchmark.png