diff --git a/fep/f011/fep-f011.abnf b/fep/f011/fep-f011.abnf new file mode 100644 index 0000000..e4a2071 --- /dev/null +++ b/fep/f011/fep-f011.abnf @@ -0,0 +1,100 @@ +// Tested with: http://lab.antlr.org/ + +grammar Query; + +// ================== Parser rules ================== + +// New explicit start rule +query + : expr EOF + ; + +expr + : or_expr + ; + + +or_expr + : and_expr (OR and_expr)* + ; + +and_expr + : unary ((AND unary) | unary)* + ; + +unary + : NOT unary + | primary + ; + +primary + : term + | phrase + | regex + | range + | facet_expr + | LPAREN expr RPAREN + ; + +facet_expr + : WORD COLON (primary | LPAREN expr RPAREN) + ; + +range + : (LBRACK | LBRACE) bound TO bound (RBRACK | RBRACE) + ; + +bound + : WORD + | phrase + ; + +term + : WORD + ; + +// Parser-level wrappers for PHRASE and REGEX tokens +phrase + : PHRASE + ; + +regex + : REGEX + ; + +// ================== Lexer rules ================== + +// Keywords +AND : 'AND'; +OR : 'OR'; +NOT : 'NOT'; +TO : 'TO'; + +// Delimiters / punctuation +COLON : ':'; +LPAREN : '('; +RPAREN : ')'; +LBRACK : '['; +RBRACK : ']'; +LBRACE : '{'; +RBRACE : '}'; + +// WORD: run of non-space, non-delimiter, non-quote, non-slash characters. +WORD + : ~[ \t\r\n"/:(){}[\]]+ + ; + +// phrase := '"' * '"' +PHRASE + : '"' (~["\r\n])+ '"' + ; + +// regex := '/' '/' +REGEX + : '/' (~[/\r\n])+ '/' + ; + +// All whitespace skipped +WS + : [ \t\r\n]+ -> skip + ; diff --git a/fep/f011/fep-f011.jsonld b/fep/f011/fep-f011.jsonld new file mode 100644 index 0000000..6e11056 --- /dev/null +++ b/fep/f011/fep-f011.jsonld @@ -0,0 +1,17 @@ +{ + "@context": [ + "https://www.w3.org/ns/activitystreams", + { + "fepf011": "https://w3id.org/fep/f011#", + "endpoints": { + "@id": "https://www.w3.org/ns/activitypub#endpoints", + "@context": { + "search": { + "@id": "fepf011:search", + "@type": "@id" + } + } + } + } + ] +} \ No newline at end of file diff --git a/fep/f011/fep-f011.md b/fep/f011/fep-f011.md new file mode 100644 index 0000000..7bfe980 --- /dev/null +++ b/fep/f011/fep-f011.md @@ -0,0 +1,233 @@ +--- +slug: "f011" +authors: Steve Bate +status: DRAFT +dateReceived: 2026-03-17 +discussionsTo: https://codeberg.org/steve-bate/fep/issues +--- +# FEP-f011: Full-Text Search Query Syntax for ActivityPub + +## Summary + +This proposal defines a full-text search syntax for ActivityPub client support. + +The syntax supports: + +- Terms and quoted phrases +- Faceted search (`facet:value`) +- Boolean operators (`AND`, `OR`, `NOT`) +- Parenthesized groups +- Implicit `AND` between adjacent expressions +- Regex literals (`/.../`) +- Inclusive, exclusive, and mixed range bounds (`[a TO b]`, `{a TO b}`, `[a TO b}`) + +## Terminology + +The term *query string* means the value of the `q` query parameter in an HTTP request +against a search endpoint. + +The term *faceted expression* means `:expr`, where `expr` may be a term, +phrase, regex, range, or parenthesized subexpression and the *facet* is typically a JSON property path. + +The term *unfaceted expression* means a term, phrase, regex, or group that does not +have a facet prefix. + +## Query Syntax + +### Grammar + +The (simplified) grammar proposed by this FEP is: + +```ebnf +query := or_expr +or_expr := and_expr ("OR" and_expr)* +and_expr := unary (("AND" unary) | (implicit_and unary))* +unary := "NOT" unary | primary +primary := term | phrase | regex | range | field_expr | "(" query ")" +field_expr := WORD ":" (primary | "(" query ")") +range := ("[" | "{") bound TO bound ("]" | "}") +bound := WORD | phrase +phrase := '"' * '"' +regex := "/" "/" +term := WORD +``` + +An AND is implicit when two primaries are adjacent. The "regex-body" SHOULD conform to [RFC9485] (Interoperable Regular Expression Format). + +A full ABNF grammar is [available](fep-f011.abnf). This can be used with parser generators like [ANTLR](https://www.antlr.org/). + +### Token Rules + +- `AND`, `OR`, `NOT`, and `TO` are recognized as operators only in uppercase. +- `WORD` excludes whitespace and these delimiters: `()[]{}":/`. +- Regex literals use slash delimiters and support escaped slash characters. + +### Operator Precedence + +The parser evaluates operators in this order: + +1. Parenthesized expressions +2. Unary `NOT` +3. `AND` (explicit and implicit) +4. `OR` + +`AND` and `OR` are left-associative. + +### Faceted Expressions + +A fielded expression has this form: + +```text +:value +:"some phrase" +:/xy.*/ +:[10 TO 20] +:(cats OR dogs) +``` + +`:(...)` applies the nested expression to the facet value. + +### Range Expressions + +Ranges support inclusive and exclusive delimiters independently: + +- `[a TO b]` inclusive lower and upper bounds +- `{a TO b}` exclusive lower and upper bounds +- `[a TO b}` inclusive lower, exclusive upper +- `{a TO b]` exclusive lower, inclusive upper + +Unbounded bounds may use `*` where supported by the evaluator implementation +(e.g., `rank:[4 TO *]`). + + +## Search Endpoint Discovery + +The search endpoint is specified using the `search` property of the ActivityPub actor `endpoints` object. For example: + +```json +{ + "@context": [ + "https://w3id.org/fep/f011" + "https://www.w3.org/ns/activitystreams" + ], + ... + "endpoints": { + "search": "https://server.example/search" + } +} +``` +## HTTP Usage + +This syntax is used as the `q` parameter on the search endpoint. + +```text +GET https://server.example/search?q= +``` + +### Responses + +Responses MUST be an ActivityPub `Collection` or `OrderedCollection`. Paging is optional. If an `OrderedCollection` is returned, the results SHOULD be considered ranked although the ranking criteria will not be known by the client ( possibilities are relevance/[BM25](https://en.wikipedia.org/wiki/Okapi_BM25), recency, etc.). + +### Example 1: Simple Primary Term + +Request: `cats` + +```http +GET /search?q=cats HTTP/1.1 +Host: social.example +Accept: application/activity+json +``` + +### Example 2: Faceted + Unfaceted phrase + +Request: `tag:fediverse language:en "ActivityPub client search"` + +```http +GET /search?q=tag%3Afediverse%20language%3Aen%20%22ActivityPub%20client%20search%22 HTTP/1.1 +Host: social.example +Accept: application/activity+json +``` + +Response: + +```json +{ + "@context": "https://www.w3.org/ns/activitystreams", + "type": "OrderedCollection", + "id": "https://social.example/search?q=tag:fediverse language:en \"ActivityPub client search\"", + "totalItems": 1, + "orderedItems": [ + { + "id": "https://social.example/notes/1", + "type": "Note" + } + ] +} +``` + +### Example 3: Boolean + Grouping + +Request:v`(cats OR dogs) AND NOT testing` + + +```http +GET /search?q=%28cats%20OR%20dogs%29%20AND%20NOT%20testing HTTP/1.1 +Host: social.example +Accept: application/activity+json +``` + +### Example 4: Regex + +Request: `type:Person preferredUsername:/alic.*/` + +```http +GET /search?q=type%3APerson%20preferredUsername%3A%2Ffedi.%2A%2F +Host: social.example +Accept: application/activity+json +``` + +This could be useful for auto completion. Using a `type:HashTag` search term could similarly be used for hashtag autocompletion. + +### Example 5: Range + +Request: `rank:[2 TO 4]` + +```http +GET /search?q=rank%3A%5B2%20TO%204%5D HTTP/1.1 +Host: social.example +Accept: application/activity+json +``` + +### Error Handling + +Invalid syntax MUST result in an HTTP client error status. +Examples include: + +- Missing closing `)` +- Missing value after `:` +- Unterminated regex literal +- Missing range delimiter + +Implementations MAY include details in a problem response body. + +## Interoperability Notes + +- Operators are case-sensitive (`AND` is an operator, `and` is a term). +- Unfaceted regex and primary term evaluation depends on pre-configured text facets in the evaluation layer. +- Field capabilities are evaluator-specific +- This proposal standardizes syntax and parsing behavior, not ranking/scoring. + +## Implementations + +- [FIRM](https://github.com/steve-bate/firm) + + +## Copyright + +CC0 1.0 Universal (CC0 1.0) Public Domain Dedication + +To the extent possible under law, the authors of this Fediverse Enhancement +Proposal have waived all copyright and related or neighboring rights to this work. + +[ActivityPub]: https://www.w3.org/TR/activitypub/ "ActivityPub is a decentralized social networking protocol based upon the ActivityStreams 2.0 data format." +[RFC9485]: https://www.rfc-editor.org/rfc/rfc9485.html#name-pcre-re2-and-ruby-regexps "This document specifies I-Regexp, a flavor of regular expression that is limited in scope with the goal of interoperation across many different regular expression libraries." \ No newline at end of file