Parking ideas for a decentralized search engine from first principles in 2020
Goal - to build a new age search engine (fun project) that is semantic and decentralized.
Some initial thoughts
- Multi-nodal desktop app
- Electron UI (or PWA)
- Central node (CN)
- There will be a central node which will maintain:
- Metadata (models)
- Views (a model can have multiple published views - like “data grid”, “catalog”, “photos”, “conversation”, “resume” etc.
- Extractors (scripts)
- Directory (of all nodes)
- There will be a central node which will maintain:
- Modeling
- The user should be able to add / edit models. A user can contribute metadata to the CN
- Semantic modeling (Bootstrap models from schema.org)
- Features:
- Ability to add / edit models
- Ability to add / edit objects
- Data Extraction
- Users should be able to write and publish “extractors”
- Source Plug-in architecture (example Wikipedia plugin, StackOverflow plugin, Reddit plugin, Amazon plugin etc.).
- Data Source
- Users should be able to store and publish objects.
- Users should also publish data-directory
- The source plugins will be extract information and map it semantically in a local SQLite database.
- View
- Users should be able to write and publish views in HTML/CSS
- Multiple object view. with plugins for rendering objects as list, graphs, images
- Single object view. with templates for each object type and also showing modeled objects with internet links for further search
- Sharing
- Ability to publish / share models, objects, extractors, views in-app
- Network
- Offline first
- Personal node first
- Personal node can be setup as a server
- Ability to create a peer network (node to node)
- Ability to search a peer via a plugin
Reading
Pears: https://pearsearch.org/
Blog on personal search: https://beepb00p.xyz/pkm-search.html