Adam Ierymenko
13 years ago
12 changed files with 50 additions and 918 deletions
@ -1,70 +0,0 @@
|
||||
kissdb |
||||
====== |
||||
|
||||
(Keep It) Simple Stupid Database |
||||
|
||||
KISSDB is about the simplest key/value store you'll ever see, anywhere. |
||||
It's written in plain vanilla C using only the standard string and FILE |
||||
I/O functions, and should port to just about anything with a disk or |
||||
something that acts like one. |
||||
|
||||
It stores keys and values of fixed length in a stupid-simple file format |
||||
based on fixed-size hash tables. If a hash collision occurrs, a new "page" |
||||
of hash table is appended to the database. The format is append-only. |
||||
There is no delete. Puts that replace an existing value, however, will not |
||||
grow the file as they will overwrite the existing entry. |
||||
|
||||
Hash table size is a space/speed trade-off parameter. Larger hash tables |
||||
will reduce collisions and speed things up a bit, at the expense of memory |
||||
and disk space. A good size is usually about 1/2 the average number of |
||||
entries you expect. |
||||
|
||||
Features: |
||||
|
||||
* Tiny, compiles to ~4k on an x86_64 Linux system |
||||
* Small memory footprint (only caches hash tables) |
||||
* Very space-efficient (on disk) if small hash tables are used |
||||
* Makes a decent effort to be robust on power loss |
||||
* Pretty respectably fast, especially given its simplicity |
||||
* 64-bit, file size limit is 2^64 bytes |
||||
* Ports to anything with a C compiler and stdlib/stdio |
||||
* Public domain |
||||
|
||||
Limitations: |
||||
|
||||
* Fixed-size keys and values, must recreate and copy to change any init size parameter |
||||
* Add/update only, no delete |
||||
* Iteration is supported but key order is undefined |
||||
* No search for subsets of keys/values |
||||
* No indexes |
||||
* No transactions |
||||
* No special recovery features if a database gets corrupted |
||||
* No built-in thread-safety (guard it with a mutex in MT code) |
||||
* No built-in caching of data (only hash tables are cached for lookup speed) |
||||
* No endian-awareness (currently), so big-endian DBs won't read on little-endian machines |
||||
|
||||
Alternative key/value stores and embedded databases: |
||||
|
||||
* [MDB](http://symas.com/mdb/) uses mmap() and is very fast (not quite as tiny/simple/portable) |
||||
* [CDB](http://cr.yp.to/cdb.html) is also minimal and fast, probably the closest thing to this (but has a 4gb size limit) |
||||
* [Kyoto Cabinet](http://fallabs.com/kyotocabinet/) is very fast, full-featured, and modern (license required for commercial use) |
||||
* [SQLite](http://www.sqlite.org/) gives you a complete embedded SQL server (public domain, very mature, much larger) |
||||
* Others include GDBM, NDBM, Berkeley DB, etc. Use your Googles. :) |
||||
|
||||
KISSDB is good if you want space-efficient relatively fast write-once/read-many storage |
||||
of keys mapped to values. It's not a good choice if you need searches, indexes, delete, |
||||
structured storage, or widely varying key/value sizes. It's also probably not a good |
||||
choice if you need a long-lived database for critical data, since it lacks recovery |
||||
features and is brittle if its internals are modified. It would be better for a cache |
||||
of data that can be restored or "re-learned," such as keys, Bitcoin transactions, nodes |
||||
on a peer-to-peer network, log analysis results, rendered web pages, session cookies, |
||||
auth tokens, etc. |
||||
|
||||
KISSDB is in the public domain. One reason it was written was the |
||||
poverty of simple key/value databases with wide open licensing. Even old |
||||
ones like GDBM have GPL, not LGPL, licenses. |
||||
|
||||
See comments in kissdb.h for documentation. Makefile can be used to build |
||||
a test program on systems with gcc. |
||||
|
||||
Author: Adam Ierymenko / ZeroTier Networks LLC |
||||
@ -1,60 +0,0 @@
|
||||
----- |
||||
|
||||
KISSDB file format (version 2) |
||||
Author: Adam Ierymenko <adam.ierymenko@zerotier.com> |
||||
|
||||
----- |
||||
|
||||
In keeping with the goal of minimalism the file format is very simple, the |
||||
sort of thing that would be given as an example in an introductory course in |
||||
data structures. It's a basic hash table that adds additional pages of hash |
||||
table entries on collision. |
||||
|
||||
It consists of a 28 byte header followed by a series of hash tables and data. |
||||
All integer values are stored in the native word order of the target |
||||
architecture (in the future the code might be fixed to make everything |
||||
little-endian if anyone cares about that). |
||||
|
||||
The header consists of the following fields: |
||||
|
||||
[0-3] magic numbers: (ASCII) 'K', 'd', 'B', KISSDB_VERSION (currently 2) |
||||
[4-11] 64-bit hash table size in entries |
||||
[12-19] 64-bit key size in bytes |
||||
[20-27] 64-bit value size in bytes |
||||
|
||||
Hash tables are arrays of [hash table size + 1] 64-bit integers. The extra |
||||
entry, if nonzero, is the offset in the file of the next hash table, forming |
||||
a linked list of hash tables across the file. |
||||
|
||||
Immediately following the header, the first hash table will be written when |
||||
the first key/value is added. The algorithm for adding new entries is as |
||||
follows: |
||||
|
||||
(1) The key is hashed using a 64-bit variant of the DJB2 hash function, and |
||||
this is taken modulo hash table size to get a bucket number. |
||||
(2) Hash tables are checked in order, starting with the first hash table, |
||||
until a zero (empty) bucket is found. If one is found, skip to step (4). |
||||
(3) If no empty buckets are found in any hash table, a new table is appended |
||||
to the file and the final pointer in the previous hash table is set to |
||||
its offset. (In the code the update of the next hash table pointer in |
||||
the previous hash table happens last, after the whole write is complete, |
||||
to avoid corruption on power loss.) |
||||
(4) The key and value are appended, in order with no additional meta-data, |
||||
to the database file. Before appending the offset in the file stream |
||||
where they will be stored is saved. After appending, this offset is |
||||
written to the empty hash table bucket we chose in steps 2/3. Hash table |
||||
updates happen last to avoid corruption if the write does not complete. |
||||
|
||||
Lookup of a key/value pair occurs as follows: |
||||
|
||||
(1) The key is hashed and taken modulo hash table size to get a bucket |
||||
number. |
||||
(2) If this bucket's entry in the hash table is nonzero, the key at the |
||||
offset specified by this bucket is compared to the key being looked up. |
||||
If they are equal, the value is read and returned. |
||||
(3) If the keys are not equal, the next hash table is checked and step (2) |
||||
is repeated. If an empty bucket is encountered or if we run out of hash |
||||
tables, the key was not found. |
||||
|
||||
To update an existing value, its location is looked up and the value portion |
||||
of the entry is rewritten. |
||||
@ -1,447 +0,0 @@
|
||||
/* (Keep It) Simple Stupid Database
|
||||
* |
||||
* Written by Adam Ierymenko <adam.ierymenko@zerotier.com> |
||||
* KISSDB is in the public domain and is distributed with NO WARRANTY. */ |
||||
|
||||
/* Compile with KISSDB_TEST to build as a test program. */ |
||||
|
||||
/* Note: big-endian systems will need changes to implement byte swapping
|
||||
* on hash table file I/O. Or you could just use it as-is if you don't care |
||||
* that your database files will be unreadable on little-endian systems. */ |
||||
|
||||
#define _FILE_OFFSET_BITS 64 |
||||
|
||||
#include "kissdb.h" |
||||
|
||||
#include <string.h> |
||||
#include <stdlib.h> |
||||
#include <stdint.h> |
||||
|
||||
#ifdef _WIN32 |
||||
#define fseeko _fseeki64 |
||||
#define ftello _ftelli64 |
||||
#endif |
||||
|
||||
#define KISSDB_HEADER_SIZE ((sizeof(uint64_t) * 3) + 4) |
||||
|
||||
/* djb2 hash function */ |
||||
static uint64_t KISSDB_hash(const void *b,unsigned long len) |
||||
{ |
||||
unsigned long i; |
||||
uint64_t hash = 5381; |
||||
for(i=0;i<len;++i) |
||||
hash = ((hash << 5) + hash) + (uint64_t)(((const uint8_t *)b)[i]); |
||||
return hash; |
||||
} |
||||
|
||||
int KISSDB_open( |
||||
KISSDB *db, |
||||
const char *path, |
||||
int mode, |
||||
unsigned long hash_table_size, |
||||
unsigned long key_size, |
||||
unsigned long value_size) |
||||
{ |
||||
uint64_t tmp; |
||||
uint8_t tmp2[4]; |
||||
uint64_t *httmp; |
||||
uint64_t *hash_tables_rea; |
||||
|
||||
#ifdef _WIN32 |
||||
db->f = (FILE *)0; |
||||
fopen_s(&db->f,path,((mode == KISSDB_OPEN_MODE_RWREPLACE) ? "w+b" : (((mode == KISSDB_OPEN_MODE_RDWR)||(mode == KISSDB_OPEN_MODE_RWCREAT)) ? "r+b" : "rb"))); |
||||
#else |
||||
db->f = fopen(path,((mode == KISSDB_OPEN_MODE_RWREPLACE) ? "w+b" : (((mode == KISSDB_OPEN_MODE_RDWR)||(mode == KISSDB_OPEN_MODE_RWCREAT)) ? "r+b" : "rb"))); |
||||
#endif |
||||
if (!db->f) { |
||||
if (mode == KISSDB_OPEN_MODE_RWCREAT) { |
||||
#ifdef _WIN32 |
||||
db->f = (FILE *)0; |
||||
fopen_s(&db->f,path,"w+b"); |
||||
#else |
||||
db->f = fopen(path,"w+b"); |
||||
#endif |
||||
} |
||||
if (!db->f) |
||||
return KISSDB_ERROR_IO; |
||||
} |
||||
|
||||
if (fseeko(db->f,0,SEEK_END)) { |
||||
fclose(db->f); |
||||
return KISSDB_ERROR_IO; |
||||
} |
||||
if (ftello(db->f) < KISSDB_HEADER_SIZE) { |
||||
/* write header if not already present */ |
||||
if ((hash_table_size)&&(key_size)&&(value_size)) { |
||||
if (fseeko(db->f,0,SEEK_SET)) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
tmp2[0] = 'K'; tmp2[1] = 'd'; tmp2[2] = 'B'; tmp2[3] = KISSDB_VERSION; |
||||
if (fwrite(tmp2,4,1,db->f) != 1) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
tmp = hash_table_size; |
||||
if (fwrite(&tmp,sizeof(uint64_t),1,db->f) != 1) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
tmp = key_size; |
||||
if (fwrite(&tmp,sizeof(uint64_t),1,db->f) != 1) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
tmp = value_size; |
||||
if (fwrite(&tmp,sizeof(uint64_t),1,db->f) != 1) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
fflush(db->f); |
||||
} else { |
||||
fclose(db->f); |
||||
return KISSDB_ERROR_INVALID_PARAMETERS; |
||||
} |
||||
} else { |
||||
if (fseeko(db->f,0,SEEK_SET)) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
if (fread(tmp2,4,1,db->f) != 1) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
if ((tmp2[0] != 'K')||(tmp2[1] != 'd')||(tmp2[2] != 'B')||(tmp2[3] != KISSDB_VERSION)) { |
||||
fclose(db->f); |
||||
return KISSDB_ERROR_CORRUPT_DBFILE; |
||||
} |
||||
if (fread(&tmp,sizeof(uint64_t),1,db->f) != 1) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
if (!tmp) { |
||||
fclose(db->f); |
||||
return KISSDB_ERROR_CORRUPT_DBFILE; |
||||
} |
||||
hash_table_size = (unsigned long)tmp; |
||||
if (fread(&tmp,sizeof(uint64_t),1,db->f) != 1) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
if (!tmp) { |
||||
fclose(db->f); |
||||
return KISSDB_ERROR_CORRUPT_DBFILE; |
||||
} |
||||
key_size = (unsigned long)tmp; |
||||
if (fread(&tmp,sizeof(uint64_t),1,db->f) != 1) { fclose(db->f); return KISSDB_ERROR_IO; } |
||||
if (!tmp) { |
||||
fclose(db->f); |
||||
return KISSDB_ERROR_CORRUPT_DBFILE; |
||||
} |
||||
value_size = (unsigned long)tmp; |
||||
} |
||||
|
||||
db->hash_table_size = hash_table_size; |
||||
db->key_size = key_size; |
||||
db->value_size = value_size; |
||||
db->hash_table_size_bytes = sizeof(uint64_t) * (hash_table_size + 1); /* [hash_table_size] == next table */ |
||||
|
||||
httmp = malloc(db->hash_table_size_bytes); |
||||
if (!httmp) { |
||||
fclose(db->f); |
||||
return KISSDB_ERROR_MALLOC; |
||||
} |
||||
db->num_hash_tables = 0; |
||||
db->hash_tables = (uint64_t *)0; |
||||
while (fread(httmp,db->hash_table_size_bytes,1,db->f) == 1) { |
||||
hash_tables_rea = realloc(db->hash_tables,db->hash_table_size_bytes * (db->num_hash_tables + 1)); |
||||
if (!hash_tables_rea) { |
||||
KISSDB_close(db); |
||||
free(httmp); |
||||
return KISSDB_ERROR_MALLOC; |
||||
} |
||||
db->hash_tables = hash_tables_rea; |
||||
|
||||
memcpy(((uint8_t *)db->hash_tables) + (db->hash_table_size_bytes * db->num_hash_tables),httmp,db->hash_table_size_bytes); |
||||
++db->num_hash_tables; |
||||
if (httmp[db->hash_table_size]) { |
||||
if (fseeko(db->f,httmp[db->hash_table_size],SEEK_SET)) { |
||||
KISSDB_close(db); |
||||
free(httmp); |
||||
return KISSDB_ERROR_IO; |
||||
} |
||||
} else break; |
||||
} |
||||
free(httmp); |
||||
|
||||
return 0; |
||||
} |
||||
|
||||
void KISSDB_close(KISSDB *db) |
||||
{ |
||||
if (db->hash_tables) |
||||
free(db->hash_tables); |
||||
if (db->f) |
||||
fclose(db->f); |
||||
memset(db,0,sizeof(KISSDB)); |
||||
} |
||||
|
||||
int KISSDB_get(KISSDB *db,const void *key,void *vbuf) |
||||
{ |
||||
uint8_t tmp[4096]; |
||||
const uint8_t *kptr; |
||||
unsigned long klen,i; |
||||
uint64_t hash = KISSDB_hash(key,db->key_size) % (uint64_t)db->hash_table_size; |
||||
uint64_t offset; |
||||
uint64_t *cur_hash_table; |
||||
long n; |
||||
|
||||
cur_hash_table = db->hash_tables; |
||||
for(i=0;i<db->num_hash_tables;++i) { |
||||
offset = cur_hash_table[hash]; |
||||
if (offset) { |
||||
if (fseeko(db->f,offset,SEEK_SET)) |
||||
return KISSDB_ERROR_IO; |
||||
|
||||
kptr = (const uint8_t *)key; |
||||
klen = db->key_size; |
||||
while (klen) { |
||||
n = (long)fread(tmp,1,(klen > sizeof(tmp)) ? sizeof(tmp) : klen,db->f); |
||||
if (n > 0) { |
||||
if (memcmp(kptr,tmp,n)) |
||||
goto get_no_match_next_hash_table; |
||||
kptr += n; |
||||
klen -= (unsigned long)n; |
||||
} else return 1; /* not found */ |
||||
} |
||||
|
||||
if (fread(vbuf,db->value_size,1,db->f) == 1) |
||||
return 0; /* success */ |
||||
else return KISSDB_ERROR_IO; |
||||
} else return 1; /* not found */ |
||||
get_no_match_next_hash_table: |
||||
cur_hash_table += db->hash_table_size + 1; |
||||
} |
||||
|
||||
return 1; /* not found */ |
||||
} |
||||
|
||||
int KISSDB_put(KISSDB *db,const void *key,const void *value) |
||||
{ |
||||
uint8_t tmp[4096]; |
||||
const uint8_t *kptr; |
||||
unsigned long klen,i; |
||||
uint64_t hash = KISSDB_hash(key,db->key_size) % (uint64_t)db->hash_table_size; |
||||
uint64_t offset; |
||||
uint64_t htoffset,lasthtoffset; |
||||
uint64_t endoffset; |
||||
uint64_t *cur_hash_table; |
||||
uint64_t *hash_tables_rea; |
||||
long n; |
||||
|
||||
lasthtoffset = htoffset = KISSDB_HEADER_SIZE; |
||||
cur_hash_table = db->hash_tables; |
||||
for(i=0;i<db->num_hash_tables;++i) { |
||||
offset = cur_hash_table[hash]; |
||||
if (offset) { |
||||
/* rewrite if already exists */ |
||||
if (fseeko(db->f,offset,SEEK_SET)) |
||||
return KISSDB_ERROR_IO; |
||||
|
||||
kptr = (const uint8_t *)key; |
||||
klen = db->key_size; |
||||
while (klen) { |
||||
n = (long)fread(tmp,1,(klen > sizeof(tmp)) ? sizeof(tmp) : klen,db->f); |
||||
if (n > 0) { |
||||
if (memcmp(kptr,tmp,n)) |
||||
goto put_no_match_next_hash_table; |
||||
kptr += n; |
||||
klen -= (unsigned long)n; |
||||
} |
||||
} |
||||
|
||||
if (fwrite(value,db->value_size,1,db->f) == 1) { |
||||
fflush(db->f); |
||||
return 0; /* success */ |
||||
} else return KISSDB_ERROR_IO; |
||||
} else { |
||||
/* add if an empty hash table slot is discovered */ |
||||
if (fseeko(db->f,0,SEEK_END)) |
||||
return KISSDB_ERROR_IO; |
||||
endoffset = ftello(db->f); |
||||
|
||||
if (fwrite(key,db->key_size,1,db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
if (fwrite(value,db->value_size,1,db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
|
||||
if (fseeko(db->f,htoffset + (sizeof(uint64_t) * hash),SEEK_SET)) |
||||
return KISSDB_ERROR_IO; |
||||
if (fwrite(&endoffset,sizeof(uint64_t),1,db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
cur_hash_table[hash] = endoffset; |
||||
|
||||
fflush(db->f); |
||||
|
||||
return 0; /* success */ |
||||
} |
||||
put_no_match_next_hash_table: |
||||
lasthtoffset = htoffset; |
||||
htoffset = cur_hash_table[db->hash_table_size]; |
||||
cur_hash_table += (db->hash_table_size + 1); |
||||
} |
||||
|
||||
/* if no existing slots, add a new page of hash table entries */ |
||||
if (fseeko(db->f,0,SEEK_END)) |
||||
return KISSDB_ERROR_IO; |
||||
endoffset = ftello(db->f); |
||||
|
||||
hash_tables_rea = realloc(db->hash_tables,db->hash_table_size_bytes * (db->num_hash_tables + 1)); |
||||
if (!hash_tables_rea) |
||||
return KISSDB_ERROR_MALLOC; |
||||
db->hash_tables = hash_tables_rea; |
||||
cur_hash_table = &(db->hash_tables[(db->hash_table_size + 1) * db->num_hash_tables]); |
||||
memset(cur_hash_table,0,db->hash_table_size_bytes); |
||||
|
||||
cur_hash_table[hash] = endoffset + db->hash_table_size_bytes; /* where new entry will go */ |
||||
|
||||
if (fwrite(cur_hash_table,db->hash_table_size_bytes,1,db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
|
||||
if (fwrite(key,db->key_size,1,db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
if (fwrite(value,db->value_size,1,db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
|
||||
if (db->num_hash_tables) { |
||||
if (fseeko(db->f,lasthtoffset + (sizeof(uint64_t) * db->hash_table_size),SEEK_SET)) |
||||
return KISSDB_ERROR_IO; |
||||
if (fwrite(&endoffset,sizeof(uint64_t),1,db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
db->hash_tables[((db->hash_table_size + 1) * (db->num_hash_tables - 1)) + db->hash_table_size] = endoffset; |
||||
} |
||||
|
||||
++db->num_hash_tables; |
||||
|
||||
fflush(db->f); |
||||
|
||||
return 0; /* success */ |
||||
} |
||||
|
||||
void KISSDB_Iterator_init(KISSDB *db,KISSDB_Iterator *dbi) |
||||
{ |
||||
dbi->db = db; |
||||
dbi->h_no = 0; |
||||
dbi->h_idx = 0; |
||||
} |
||||
|
||||
int KISSDB_Iterator_next(KISSDB_Iterator *dbi,void *kbuf,void *vbuf) |
||||
{ |
||||
uint64_t offset; |
||||
|
||||
if ((dbi->h_no < dbi->db->num_hash_tables)&&(dbi->h_idx < dbi->db->hash_table_size)) { |
||||
while (!(offset = dbi->db->hash_tables[((dbi->db->hash_table_size + 1) * dbi->h_no) + dbi->h_idx])) { |
||||
if (++dbi->h_idx >= dbi->db->hash_table_size) { |
||||
dbi->h_idx = 0; |
||||
if (++dbi->h_no >= dbi->db->num_hash_tables) |
||||
return 0; |
||||
} |
||||
} |
||||
if (fseeko(dbi->db->f,offset,SEEK_SET)) |
||||
return KISSDB_ERROR_IO; |
||||
if (fread(kbuf,dbi->db->key_size,1,dbi->db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
if (fread(vbuf,dbi->db->value_size,1,dbi->db->f) != 1) |
||||
return KISSDB_ERROR_IO; |
||||
if (++dbi->h_idx >= dbi->db->hash_table_size) { |
||||
dbi->h_idx = 0; |
||||
++dbi->h_no; |
||||
} |
||||
return 1; |
||||
} |
||||
|
||||
return 0; |
||||
} |
||||
|
||||
#ifdef KISSDB_TEST |
||||
|
||||
#include <inttypes.h> |
||||
|
||||
int main(int argc,char **argv) |
||||
{ |
||||
uint64_t i,j; |
||||
uint64_t v[8]; |
||||
KISSDB db; |
||||
KISSDB_Iterator dbi; |
||||
char got_all_values[10000]; |
||||
int q; |
||||
|
||||
printf("Opening new empty database test.db...\n"); |
||||
|
||||
if (KISSDB_open(&db,"test.db",KISSDB_OPEN_MODE_RWREPLACE,1024,8,sizeof(v))) { |
||||
printf("KISSDB_open failed\n"); |
||||
return 1; |
||||
} |
||||
|
||||
printf("Adding and then re-getting 10000 64-byte values...\n"); |
||||
|
||||
for(i=0;i<10000;++i) { |
||||
for(j=0;j<8;++j) |
||||
v[j] = i; |
||||
if (KISSDB_put(&db,&i,v)) { |
||||
printf("KISSDB_put failed (%"PRIu64")\n",i); |
||||
return 1; |
||||
} |
||||
memset(v,0,sizeof(v)); |
||||
if ((q = KISSDB_get(&db,&i,v))) { |
||||
printf("KISSDB_get (1) failed (%"PRIu64") (%d)\n",i,q); |
||||
return 1; |
||||
} |
||||
for(j=0;j<8;++j) { |
||||
if (v[j] != i) { |
||||
printf("KISSDB_get (1) failed, bad data (%"PRIu64")\n",i); |
||||
return 1; |
||||
} |
||||
} |
||||
} |
||||
|
||||
printf("Getting 10000 64-byte values...\n"); |
||||
|
||||
for(i=0;i<10000;++i) { |
||||
if ((q = KISSDB_get(&db,&i,v))) { |
||||
printf("KISSDB_get (2) failed (%"PRIu64") (%d)\n",i,q); |
||||
return 1; |
||||
} |
||||
for(j=0;j<8;++j) { |
||||
if (v[j] != i) { |
||||
printf("KISSDB_get (2) failed, bad data (%"PRIu64")\n",i); |
||||
return 1; |
||||
} |
||||
} |
||||
} |
||||
|
||||
printf("Closing and re-opening database in read-only mode...\n"); |
||||
|
||||
KISSDB_close(&db); |
||||
|
||||
if (KISSDB_open(&db,"test.db",KISSDB_OPEN_MODE_RDONLY,1024,8,sizeof(v))) { |
||||
printf("KISSDB_open failed\n"); |
||||
return 1; |
||||
} |
||||
|
||||
printf("Getting 10000 64-byte values...\n"); |
||||
|
||||
for(i=0;i<10000;++i) { |
||||
if ((q = KISSDB_get(&db,&i,v))) { |
||||
printf("KISSDB_get (3) failed (%"PRIu64") (%d)\n",i,q); |
||||
return 1; |
||||
} |
||||
for(j=0;j<8;++j) { |
||||
if (v[j] != i) { |
||||
printf("KISSDB_get (3) failed, bad data (%"PRIu64")\n",i); |
||||
return 1; |
||||
} |
||||
} |
||||
} |
||||
|
||||
printf("Iterator test...\n"); |
||||
|
||||
KISSDB_Iterator_init(&db,&dbi); |
||||
i = 0xdeadbeef; |
||||
memset(got_all_values,0,sizeof(got_all_values)); |
||||
while (KISSDB_Iterator_next(&dbi,&i,&v) > 0) { |
||||
if (i < 10000) |
||||
got_all_values[i] = 1; |
||||
else { |
||||
printf("KISSDB_Iterator_next failed, bad data (%"PRIu64")\n",i); |
||||
return 1; |
||||
} |
||||
} |
||||
for(i=0;i<10000;++i) { |
||||
if (!got_all_values[i]) { |
||||
printf("KISSDB_Iterator failed, missing value index %"PRIu64"\n",i); |
||||
return 1; |
||||
} |
||||
} |
||||
|
||||
KISSDB_close(&db); |
||||
|
||||
printf("All tests OK!\n"); |
||||
|
||||
return 0; |
||||
} |
||||
|
||||
#endif |
||||
@ -1,171 +0,0 @@
|
||||
/* (Keep It) Simple Stupid Database
|
||||
* |
||||
* Written by Adam Ierymenko <adam.ierymenko@zerotier.com> |
||||
* KISSDB is in the public domain and is distributed with NO WARRANTY. */ |
||||
|
||||
#ifndef ___KISSDB_H |
||||
#define ___KISSDB_H |
||||
|
||||
#include <stdio.h> |
||||
#include <stdint.h> |
||||
|
||||
#ifdef __cplusplus |
||||
extern "C" { |
||||
#endif |
||||
|
||||
/**
|
||||
* Version: 2 |
||||
* |
||||
* This is the file format identifier, and changes any time the file |
||||
* format changes. The code version will be this dot something, and can |
||||
* be seen in tags in the git repository. |
||||
*/ |
||||
#define KISSDB_VERSION 2 |
||||
|
||||
/**
|
||||
* KISSDB database state |
||||
* |
||||
* These fields can be read by a user, e.g. to look up key_size and |
||||
* value_size, but should never be changed. |
||||
*/ |
||||
typedef struct { |
||||
unsigned long hash_table_size; |
||||
unsigned long key_size; |
||||
unsigned long value_size; |
||||
unsigned long hash_table_size_bytes; |
||||
unsigned long num_hash_tables; |
||||
uint64_t *hash_tables; |
||||
FILE *f; |
||||
} KISSDB; |
||||
|
||||
/**
|
||||
* I/O error or file not found |
||||
*/ |
||||
#define KISSDB_ERROR_IO -1 |
||||
|
||||
/**
|
||||
* Out of memory |
||||
*/ |
||||
#define KISSDB_ERROR_MALLOC -2 |
||||
|
||||
/**
|
||||
* Invalid paramters (e.g. missing _size paramters on init to create database) |
||||
*/ |
||||
#define KISSDB_ERROR_INVALID_PARAMETERS -3 |
||||
|
||||
/**
|
||||
* Database file appears corrupt |
||||
*/ |
||||
#define KISSDB_ERROR_CORRUPT_DBFILE -4 |
||||
|
||||
/**
|
||||
* Open mode: read only |
||||
*/ |
||||
#define KISSDB_OPEN_MODE_RDONLY 1 |
||||
|
||||
/**
|
||||
* Open mode: read/write |
||||
*/ |
||||
#define KISSDB_OPEN_MODE_RDWR 2 |
||||
|
||||
/**
|
||||
* Open mode: read/write, create if doesn't exist |
||||
*/ |
||||
#define KISSDB_OPEN_MODE_RWCREAT 3 |
||||
|
||||
/**
|
||||
* Open mode: truncate database, open for reading and writing |
||||
*/ |
||||
#define KISSDB_OPEN_MODE_RWREPLACE 4 |
||||
|
||||
/**
|
||||
* Open database |
||||
* |
||||
* The three _size parameters must be specified if the database could |
||||
* be created or re-created. Otherwise an error will occur. If the |
||||
* database already exists, these parameters are ignored and are read |
||||
* from the database. You can check the struture afterwords to see what |
||||
* they were. |
||||
* |
||||
* @param db Database struct |
||||
* @param path Path to file |
||||
* @param mode One of the KISSDB_OPEN_MODE constants |
||||
* @param hash_table_size Size of hash table in 64-bit entries (must be >0) |
||||
* @param key_size Size of keys in bytes |
||||
* @param value_size Size of values in bytes |
||||
* @return 0 on success, nonzero on error |
||||
*/ |
||||
extern int KISSDB_open( |
||||
KISSDB *db, |
||||
const char *path, |
||||
int mode, |
||||
unsigned long hash_table_size, |
||||
unsigned long key_size, |
||||
unsigned long value_size); |
||||
|
||||
/**
|
||||
* Close database |
||||
* |
||||
* @param db Database struct |
||||
*/ |
||||
extern void KISSDB_close(KISSDB *db); |
||||
|
||||
/**
|
||||
* Get an entry |
||||
* |
||||
* @param db Database struct |
||||
* @param key Key (key_size bytes) |
||||
* @param vbuf Value buffer (value_size bytes capacity) |
||||
* @return -1 on I/O error, 0 on success, 1 on not found |
||||
*/ |
||||
extern int KISSDB_get(KISSDB *db,const void *key,void *vbuf); |
||||
|
||||
/**
|
||||
* Put an entry (overwriting it if it already exists) |
||||
* |
||||
* In the already-exists case the size of the database file does not |
||||
* change. |
||||
* |
||||
* @param db Database struct |
||||
* @param key Key (key_size bytes) |
||||
* @param value Value (value_size bytes) |
||||
* @return -1 on I/O error, 0 on success |
||||
*/ |
||||
extern int KISSDB_put(KISSDB *db,const void *key,const void *value); |
||||
|
||||
/**
|
||||
* Cursor used for iterating over all entries in database |
||||
*/ |
||||
typedef struct { |
||||
KISSDB *db; |
||||
unsigned long h_no; |
||||
unsigned long h_idx; |
||||
} KISSDB_Iterator; |
||||
|
||||
/**
|
||||
* Initialize an iterator |
||||
* |
||||
* @param db Database struct |
||||
* @param i Iterator to initialize |
||||
*/ |
||||
extern void KISSDB_Iterator_init(KISSDB *db,KISSDB_Iterator *dbi); |
||||
|
||||
/**
|
||||
* Get the next entry |
||||
* |
||||
* The order of entries returned by iterator is undefined. It depends on |
||||
* how keys hash. |
||||
* |
||||
* @param Database iterator |
||||
* @param kbuf Buffer to fill with next key (key_size bytes) |
||||
* @param vbuf Buffer to fill with next value (value_size bytes) |
||||
* @return 0 if there are no more entries, negative on error, positive if an kbuf/vbuf have been filled |
||||
*/ |
||||
extern int KISSDB_Iterator_next(KISSDB_Iterator *dbi,void *kbuf,void *vbuf); |
||||
|
||||
#ifdef __cplusplus |
||||
} |
||||
#endif |
||||
|
||||
#endif |
||||
|
||||
Loading…
Reference in new issue