FlatHashMap

Hash table/map with in-place open-addressing, storing keys of type K and values of type V.

Keys are immutable except for when they are classes in which case they are head-const (through bin reinterpretation to KeyValueType), This can be overridden by setting keyEqualPred to, for instance, a == b for class keys.

Uses quadratic probing (using triangular numbers) unless usePrimeCapacity in which case a simpler probing is used.

Deletion/Removal of elements is lazy via the bitmap _holesPtr or through assignment of of reserved value of KeyType when KeyType has hole-support via trait isHoleable.

Element iteration via - either byKey, byValue or byKeyValue over FlatHashMap and - byElement over FlatHashset respects taking the container argument either as an l-value or r-value using detected using auto ref-qualified parameter introspected using (__traits(isRef, y)). In the r-value case no reference counting is needed. In the l-value case setting borrowChecked to true adds run-time support for dynamic Rust-style ownership and borrowing between the range and the container.

@safe
struct FlatHashMap (
K
V = void
alias hasher = hashOf
string keyEqualPred = defaultKeyEqualPredOf!(K)
Allocator = Mallocator
bool borrowChecked = false
bool useSmallLinearSearch = true
bool usePrimeCapacity = false
) if (
isNullable!K &&
isAllocator!Allocator
) {}

Constructors

this
this(T element)

Make with the element element.

Destructor

~this
~this()

Destruct.

Postblit

this(this)
this(this)

No copying.

Members

Aliases

ElementType
alias ElementType = T
Undocumented in source.
Flags
alias Flags = BitFlags!Flag
Undocumented in source.
KeyType
alias KeyType = K

Type of key stored.

Nullifier
alias Nullifier = typeof(K.init.nullifier)
Undocumented in source.
StoreK
alias StoreK = void*
Undocumented in source.
StoreK
alias StoreK = void*
Undocumented in source.
StoreK
alias StoreK = K
Undocumented in source.
T
alias T = K
Undocumented in source.
ValueType
alias ValueType = V

Type of value stored.

keyEqualPredFn
alias keyEqualPredFn = binaryFun!keyEqualPred
Undocumented in source.
keyEqualPredFn
alias keyEqualPredFn = keyEqualPred
Undocumented in source.

Enums

Flag
enum Flag
Undocumented in source.
InsertionStatus
enum InsertionStatus

Map insertion status.

InsertionStatus
enum InsertionStatus

Set insertion status.

Functions

averageProbeCount
double averageProbeCount()
clear
void clear()

Empty.

contains
bool contains(SomeKey key)

Check if element is stored.

containsUsingLinearSearch
bool containsUsingLinearSearch(SomeKey key)

Check if element is stored.

containsWithHoleMoving
bool containsWithHoleMoving(K key)

Check if element is stored. Move found element to a hole if possible.

dup
typeof(this) dup()
empty
bool empty()

Check if empty.

get
inout(V) get(K key, inout(V) defaultValue)

Get value of key or defaultValue if key not present (and therefore nothrow).

getKeyRef
const(K) getKeyRef(SomeKey key, const(K) defaultKey)

Get reference to key-part of stored element at key, if present, otherwise return defaultKey.

insert
InsertionStatus insert(T element)

Insert element, being either a key-value (map-case) or a just a key (set-case).

insert
InsertionStatus insert(K key, V value)

Insert or replace value at key.

insertAndReturnElement
T insertAndReturnElement(SomeElement element)

Insert element, being either a key-value (map-case) or a just a key (set-case).

insertN
void insertN(R elements)

Insert elements, all being either a key-value (map-case) or a just a key (set-case).

opBinaryRight
const(K)* opBinaryRight(SomeKey key)
Undocumented in source. Be warned that the author may not have intended to support it.
opBinaryRight
inout(V)* opBinaryRight(SomeKey key)
Undocumented in source. Be warned that the author may not have intended to support it.
opEquals
bool opEquals(typeof(this) rhs)

Equality.

opIndex
inout(V) opIndex(SomeKey key)

Indexing.

opIndexAssign
V opIndexAssign(V value, K key)

Supports the syntax aa[key] = value;.

opIndexOpAssign
V opIndexOpAssign(Rhs rhs, K key)
Undocumented in source. Be warned that the author may not have intended to support it.
opOpAssign
typeof(this) opOpAssign(SomeKey key)
Undocumented in source. Be warned that the author may not have intended to support it.
rawStore
inout(T)[] rawStore()

Unsafe access to raw store.

rehashingRemoveN
size_t rehashingRemoveN(Keys keys)

Remove all elements matching keys followed by a rehash.

remove
bool remove(SomeKey key)

Remove element.

reserveExtra
void reserveExtra(size_t extraCapacity)

Reserve rom for extraCapacity number of extra buckets.

totalProbeCount
size_t totalProbeCount()
tryGetElementFromCtorParams
const(Class) tryGetElementFromCtorParams(Params params)

Try to retrieve class-element of type Class constructed with parameters params.

Manifest constants

growScaleP
enum growScaleP;

Numerator for grow scale.

growScaleQ
enum growScaleQ;

Denominator for grow scale.

hasAddressLikeKey
enum hasAddressLikeKey;

Is true iff K is an address, in which case holes are represented by a specific value holeKeyConstant.

hasHoleableKey
enum hasHoleableKey;
Undocumented in source.
hasHoleableKey
enum hasHoleableKey;
Undocumented in source.
hasHoleableKey
enum hasHoleableKey;
Undocumented in source.
hasHoleableKey
enum hasHoleableKey;
Undocumented in source.
hasHoleableKey
enum hasHoleableKey;
Undocumented in source.
hasValue
enum hasValue;

In the hash map case, V is non-void, and a value is stored alongside the key of type K.

holeKeyAddress
enum holeKeyAddress;
Undocumented in source.
holeKeyOffset
enum holeKeyOffset;
Undocumented in source.
isBorrowChecked
enum isBorrowChecked;

Is true iff this is borrow-checked.

nullKeyElement
enum nullKeyElement;
Undocumented in source.
nullKeyElement
enum nullKeyElement;
Undocumented in source.
passElementByValue
enum passElementByValue;

TODO remove when https://github.com/dlang/dmd/pull/11000 has been merged

Properties

binCount
size_t binCount [@property getter]

Get bin count.

length
size_t length [@property getter]

Get length (read-only).

Static functions

holeKeyConstant
K holeKeyConstant()
holeKeyConstant
K holeKeyConstant()
Undocumented in source.
holeKeyConstant
K holeKeyConstant()
Undocumented in source. Be warned that the author may not have intended to support it.
isHoleKeyConstant
bool isHoleKeyConstant(K key)
isHoleKeyConstant
bool isHoleKeyConstant(K key)
Undocumented in source. Be warned that the author may not have intended to support it.
isHoleKeyConstant
bool isHoleKeyConstant(K key)
Undocumented in source.
isOccupiedBin
bool isOccupiedBin(T bin)
Undocumented in source. Be warned that the author may not have intended to support it.
keyOf
inout(K) keyOf(inout(SomeElement) element)

Get key part of element.

keyOf
inout(K) keyOf(inout(SomeElement) element)

Element type.

keyOf
inout(K) keyOf(inout(KeyValueType) element)

Get key part.

keyOf
inout(SomeElement) keyOf(inout(SomeElement) element)

Get key part of element.

valueOf
inout(V) valueOf(inout(T) element)

Get value part of element.

valueOf
inout(V) valueOf(inout(T) element)

Element type.

withCapacity
typeof(this) withCapacity(size_t minimumCapacity)

Make with room for storing at least minimumCapacity number of elements.

withElements
typeof(this) withElements(R elements)

Make with the elements elements.

Static variables

borrowedErrorMessage
auto borrowedErrorMessage;
Undocumented in source.

Structs

T
struct T

Mutable element reference with mutable constant key and value.

Templates

isScopedKeyType
template isScopedKeyType(SomeKey)

Is true if an instance of SomeKey that can be implictly cast to K.

Variables

assumeNonFullHaystack
enum bool assumeNonFullHaystack;

Setting this true doesn't give measurable speedups so set it to false for now.

growInPlaceFlag
enum bool growInPlaceFlag;

Is true iff in-place rehashing during growth should be performed.

Parameters

K

key type

V

value type

hasher

hash function or std.digest Hash

Allocator

memory allocator for bin array

borrowChecked

only activate when it's certain that this won't be moved via std.algorithm.mutation.move()

useSmallLinearSearch

Use linear search instead probing when _store is smaller than linearSearchMaxSize

usePrimeCapacity

Use prime numbers as capacity of hash table enabling better performance of simpler hash-functions

See Also

https://github.com/abseil/abseil-cpp/blob/master/absl/container/flat_hash_map.h

https://www.sebastiansylvan.com/post/robin-hood-hashing-should-be-your-default-hash-table-implementation/

https://arxiv.org/abs/1605.04031

https://github.com/Tessil/robin-map

https://github.com/martinus/robin-hood-hashing

https://probablydance.com/2017/02/26/i-wrote-the-fastest-hashtable/

https://en.wikipedia.org/wiki/Lazy_deletion

https://forum.dlang.org/post/ejqhcsvdyyqtntkgzgae@forum.dlang.org https://gankro.github.io/blah/hashbrown-insert/

TODO: Add support for opApply by copying opApply members in ~/.dub/packages/emsi_containers-0.8.0/emsi_containers/src/containers/hashmap.d

TODO: Add bool isSorted that adds a set of indexes to the insertion and removal order of the indexes. Insertion is fast. Removal is slow. These indexes are jointly allocated with the buckets

TODO: Replace withCapacity with void capacity(size_t) like D arrays.

TODO: Replace remove branching on passElementByValue when https://github.com/dlang/dmd/pull/11000 has been merged and use in instead

TODO: Implement foreach (const k, const v; _) support. See https://forum.dlang.org/post/tj700o$1412$1@digitalmars.com

TODO: Disable pragma(inline, true) and rebenchmark

TODO: tests fails when useSmallLinearSearch is set to false

TODO: Use set of Flags (defined here) as template params

TODO: Robin-hood case introspect key type for storage of parts of hash alongside nullValue and holeValue. Typically for address-types that doesn't need scanning. Example is Address for implementation of GC. This is a D showcase for code that is difficult to write in C++.

TODO: group nxt.probing functions in Prober struct given as type template param to FlatHashMap

TODO: Make load factor dependent on current capacity or length and perhaps also type and hash-function to get memory efficiency when it matters. Similar to what is recommended in https://ticki.github.io/blog/horrible/.

TODO: For copyable types replace auto ref with logic that only passes by ref when it's faster to do so. See_Also: https://github.com/dlang/dmd/pull/11000. When -preview=in has been made the default this complexity can be removed. Or replace ref with in.

TODO: Remove use of static if (__traits(isCopyable, ...)) in cases where compiler can handle more moves

TODO: Use mmap allocator when _store.sizeof is larger than at least 8 pages

TODO: Use StoreK in store and cast between it and KeyType

TODO: Allocate _holesPtr array together with _store to reduce size of FlatHashMap to 3 words when element type doesn't support it

TODO: Fix bug in growInPlaceWithCapacity and benchmark

TODO: Modify existing unittest for struct Rel { const string name; }

TODO: Use allocator.dispose() instead of allocator.deallocate() as in https://github.com/dlang-community/containers

TODO: if hash-function is cast(size_t)(classInstance) always use prime length and shift pointer before hash based on alignof (might not be needed when module prime) to maximize memory locality when adding successively allocated pointers

TODO: Add extractElement that moves it out similar to http://en.cppreference.com/w/cpp/container/unordered_set/extract

TODO: Add merge or union algorithm here or into container/common.d. See also: http://en.cppreference.com/w/cpp/container/unordered_set/merge. this algorithm moves elements from source if they are not already in this

TODO: Robin-Hood-hashing

TODO: Enable borrowChecked unconditionally in version(debug) if and when opPostMove is implemented, in which case opPostMove() should assert false if this is borrowed. See: https://github.com/dlang/DIPs/pull/109

TODO: Save one word by making _store.length be inferred by prime_modulo.primeConstants[_primeIndex] if this is not too costly.

TODO: Only add one extra element to capacity when assumeNonFullHaystack is true

Meta