Dowser. OpenSearch. Codec
(Dowser.OpenSearch v0.4.0)
View Source
Casts OpenSearch documents and search results to and from native Elixir
terms, based on each document's own index mapping — a Dowser.Client
:decoder and :encoder:
config :my_app, MyApp.Search,
endpoint: "http://localhost:9200",
decoder: Dowser.OpenSearch.Codec,
encoder: Dowser.OpenSearch.CodecOnce configured, every API function in Dowser.OpenSearch.Document and
Dowser.OpenSearch.Search casts automatically — no per-call option needed —
and a call that passes :struct or :mapping casts even on a repo without
it. Mappings are fetched (and cached) through Dowser.OpenSearch.MappingCacher,
which the application starts.
The casting itself is dowser_client's, shared with the other backend:
Dowser.Codec documents the options and what happens when a mapping can't be
fetched, Dowser.Source how a document is walked, and Dowser.Type how each
value is cast. This module is OpenSearch's configuration of it:
- mappings come from
Dowser.OpenSearch.MappingCacherunless:mappingsays otherwise; :types,:unknown_fieldsand:mapping_failuredefault to the:dowser_opensearchapplication environment (config :dowser_opensearch, types: MyApp.Types, mapping_failure: :warn);- a failed fetch emits
[:dowser_opensearch, :mapping, :failure]; - a
flat_objectfield is opaque — see below.
Subtrees the mapping declares opaque
A flat_object field (OpenSearch's name for Elasticsearch's flattened),
and an object mapped "enabled": false, hold whatever the document put in
them — the mapping enumerates none of it. Both are returned exactly as they
arrived: values uncast, and keys left as strings rather than run through
opts[:key_fn].
That last part matters under keys: :atoms. Atoms are never garbage
collected and the table is capped (:erlang.system_info(:atom_limit), a
little over a million by default); atomizing keys that come from document
content rather than from the mapping turns any writer into a way to exhaust
it and bring the node down. A mapped field's name is one of a finite set, so
casting it is safe; a flat_object field's keys are not.
A mixed result is the price: %{title: "hi", roster: %{"Managed Care Biller" => "Sam"}}. The alternative is a cast that is unsafe by construction.
Note that this is narrower than the whole risk. keys: :atoms also casts
keys the mapping simply doesn't mention — an unmapped field under
"dynamic": false, or one added since the mapping was cached. Where a body
is wholly untrusted, keys: :atoms! (String.to_existing_atom/1) is the
option that cannot grow the table at all.
The casting options
Five options decide how documents are cast; each can be passed on any API call, given to the codec in a repo's configuration, or set for the whole application — most specific first:
Dowser.OpenSearch.Search.search(query, index: "posts", types: MyApp.Types)
config :my_app, MyApp.Search,
decoder: {Dowser.OpenSearch.Codec, types: MyApp.Types},
encoder: {Dowser.OpenSearch.Codec, types: MyApp.Types}
config :dowser_opensearch, types: MyApp.Types:mapping— where the mapping comes from: a map, aDowser.Mappingmodule, or a document module. Defaults toDowser.OpenSearch.MappingCacher, the cluster's mapping. Documents come back as maps.:struct— a document module (Dowser.Schema): every document is loaded into its struct, and its struct dumped back, against its own mapping. Per call only.:types— theDowser.Typemodule leaf values are cast through (defaultDowser.Type); seeDowser.Typefor writing one.:unknown_fields—:keep(default) or:raiseaDowser.UnknownFieldErrorfor a field the mapping doesn't declare.:mapping_failure—:error(default),:warnor:ignorewhen the mapping can't be fetched; seeDowser.Codec.
:mapping and :struct ask for casting outright: on a call to a repo with
no :decoder/:encoder configured, they bring this codec in for that call.
Summary
Functions
Casts every document found in body, returning the decoded term; see
Dowser.Codec.decode/3.
Casts source against the mapping of opts[:index], returning the encoded
term; see Dowser.Codec.encode/3.
Casts the payloads of a Dowser.OpenSearch.Document.bulk/2 operation list;
see Dowser.Codec.encode_bulk/3.
Functions
Casts every document found in body, returning the decoded term; see
Dowser.Codec.decode/3.
Casts source against the mapping of opts[:index], returning the encoded
term; see Dowser.Codec.encode/3.
Casts the payloads of a Dowser.OpenSearch.Document.bulk/2 operation list;
see Dowser.Codec.encode_bulk/3.