Dowser.
View Source
Per-node cache of OpenSearch index mappings.
A GenServer owns an ETS table; reads go straight to ETS (concurrent, lock-free), and only misses/expired entries go through the GenServer — which de-duplicates concurrent fetches for the same key (single-flight).
Entries are keyed by {endpoint, scope, index}, so the same index reached
through different contexts is cached separately, and each fetch honours its
context. The scope is what distinguishes two contexts pointing at the same
endpoint — see key/2.
Options (start_link/1)
:ttl— entry lifetime in ms (default 5 min).:sweep_interval— how often expired entries are actively purged, in ms (default 1 min).nil/0disables active sweeping; lazy expiration on read still applies.:fetch—(context, index -> {:ok, value} | {:error, reason}), how a mapping is loaded on a miss. Defaults to aDowser.Client_mappingcall. Have it return the compiled schema rather than the raw mapping to keep cached values small.:eager— preload at startup viahandle_continue/2:false(default, lazy), a list of{context, index}, or a 0-arity fun returning one.
Whether the cacher is started at all (and with which options) is decided by the supervisor — see Dowser.Opensearch.Application.
Mappings that are never fetched
A mapping given in the application environment answers for its index without any request at all — and so can never fail to be fetched:
config :dowser_opensearch,
mappings: %{"posts" => %{"properties" => %{"published_at" => %{"type" => "date"}}}}It wins over the cache. Use it for an index whose mapping is pinned, and in
test suites, where it makes casting behave as it does in production without
a cluster to ask (put/3 does the same through a running cacher).
When a fetch fails
lookup/2 distinguishes "there is no mapping" from "the mapping could not be
fetched"; fetch/2 is the lenient read that folds both into nil. The
difference matters because the second silently changes the type of what a
caller gets back — see Dowser.Opensearch.Codec's :mapping_failure.
Summary
Functions
Returns a specification to start this module under a supervisor.
Clears the whole cache.
Like lookup/2, but returns the mapping directly, or nil when there is
none to be had or the fetch failed.
Returns the mapping for {context, index}, fetching lazily on a miss.
Invalidates a single {context, index} entry.
The cache key for {context, index}: {endpoint, scope, index}.
Caches mapping for index directly, without fetching anything.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.
Clears the whole cache.
@spec fetch(Dowser.Client.Context.ref(), term()) :: map() | nil
Like lookup/2, but returns the mapping directly, or nil when there is
none to be had or the fetch failed.
The lenient read: a mapping that can't be resolved degrades a cast to
identity rather than failing the request. Dowser.Opensearch.Codec uses it
only under mapping_failure: :ignore.
Returns the mapping for {context, index}, fetching lazily on a miss.
Invalidates a single {context, index} entry.
@spec key(Dowser.Client.Context.t(), term()) :: {String.t(), binary() | nil, term()}
The cache key for {context, index}: {endpoint, scope, index}.
Two contexts pointing at the same endpoint do not necessarily see the same mapping — different credentials can mean field-level security hiding fields, or an alias resolving to a different concrete index — so the credentials are part of what identifies an entry.
They are hashed into it rather than stored. A cache key lives in an ETS
table for the lifetime of the entry, where any process can read it and a
crash dump would carry it; Dowser.Client.Context goes as far as redacting
:auth from its own Inspect, and putting it in a table here would undo
that.
The hash is SHA-256, truncated to 128 bits. An application that builds
contexts from end-user credentials makes the hashed value attacker-
influenced, and a collision there would serve one tenant another's mapping —
so the cheaper :erlang.phash2/1 is the wrong tool: its default range is
2^27, which a targeted collision search exhausts in seconds.
:http_opts is hashed alongside :auth, since a credential can also arrive
as a header and a proxy or TLS setting can change which cluster answers. The
purely client-side fields (:profile, :keys, :decoder, :encoder) are
not: they can't change what OpenSearch returns. A plain unauthenticated
context hashes to nil, so the common key stays readable.
@spec lookup(Dowser.Client.Context.ref(), term()) :: {:ok, map() | nil} | {:error, term()}
Like get/2, but tells a mapping that cannot exist apart from one that
could not be fetched — which get/2 reports the same way, and which the
caller must not: casting against no mapping is correct for the first and
silently wrong for the second.
{:ok, mapping}— a static mapping, a cached one, or a freshly fetched one.{:ok, nil}— there is no mapping to be had: no index named, or no cacher running (an application that doesn't cast at all).{:error, reason}— the fetch failed. The mapping may well exist; this request just couldn't see it.Dowser.Opensearch.Codecturns this into aDowser.Opensearch.MappingErrorby default, rather than casting nothing and handing back raw JSON values.
A static mapping (config :dowser_opensearch, mappings: %{...}) wins over
the cache and is never fetched, so it cannot fail.
Caches mapping for index directly, without fetching anything.
The seam tests use, so that casting behaves the way it does in production instead of degrading to identity for want of a cluster to ask:
setup do
Dowser.Opensearch.MappingCacher.put("posts", %{"properties" => %{...}})
endOptions
:context— the context the entry belongs to, as inkey/2(defaultnil, the unauthenticated one).:ttl— entry lifetime in ms (default:infinity, unlike a fetched entry: an entry put by hand is not a cached answer that can go stale).
For a mapping that is always known — a pinned index, a test suite that
never starts the cacher — config :dowser_opensearch, mappings: %{...}
needs no running cacher at all.