beepb00p

The holy grail [https://beepb00p.xyz/pkm-search.html#future] of this really resonated with me and fully mirrors what I've been thinking about the past few months. In my observations, it's input capture, information organization, and subsequent retrieval:

Information Capture:

Input Capture - You’re going to have all-encompassing tracking and recording of all activity, but want configurable privacy on the extent to which you want your daily conversations and observations of external things you encounter and are exposed to. Capturing input needs to be holistic and incorporate all properties of encounters and new information.

Potential sources of input:

Vision — point of view recording, see snapchat spectacles, etc as primitive examples.
Audio (voice notes and multi-party conversations) - voice calls, video, etc. and other forms of audio transmission where there is more than a single party in the interaction.
Digital interactions
You will need to keep track of web pages you visit at what times
Conversations you see on Twitter, etc.

Properties and cues must be extrapolated from the information that is captured on input, in the case of audio, transcriptions are sufficient for transcription and retrieval purposes, however since video is a visual medium, it includes significantly more properties that need to be accounted for.

The aim here is to identify sufficient data points (cues) that are subsequently represented in such a way that they are easy to search across things you…

I want to build a browser that captures everything I saw on the internet, allows me to search it, run graph algorithms (like PageRank). Improves navigation (by showing trails as tree instead of tabs). Heavily offline focused (Backend only for updates, maybe for analytics).

Difference with rewind.ai: linkkraft does not have funding, i'm solo, no apps & image/video/audio recognition. Focus on web, trails, research and using web copies, selections/highlights as part of your notes & whiteboards. Preserving all possible graphs.

My inspirations: https://pages.gseis.ucla.edu/faculty/bates/berrypicking.html, https://beepb00p.xyz/promnesia.html, Jeff Raskin (Global Search, Zoom UI) https://linkkraft.com/notes/backstory

I've built a prototype with trails tree & HTML snapshoting. For each my step even inside SPA linkkraft creates HTML snapshot.
https://arestov.github.io/linkkraft-notes/comparing/linkkraft-vs-chrome.mp4
https://arestov.github.io/linkkraft-notes/trails-tree-plus-offline-for-tool-for-thoughts/linkkraft-fresh-offline-spa.mp4

Across similar software ("too many tabs" solvers, snapshoters) combination of trails tree & snapshots that reinforces is unique in linkkraft.

To make snapshot it takes current state of DOM, current state of styles (to capture CSSinJS stuff), take resources like font/images from network storage (Chrome Dev Tools protocol) and saves it as HTML on disk.

I've been developing created frontend in-memory graph DB / state manager for 1) replication 2)…

Yes, this siloing stops me from using annotation features "native" to each app and format as such as well.

Instead, I prefer to exfiltrate information from the silos(apps, formats, etc) and put them into my note taking system. Then I can do highlights, annotations, etc. on my own terms and also get the benefits of centralization such as searching and linking(the OP has another post describing their own system, which is pretty cool[1]). Currently I'm using Notion, which is also a silo of its own, but it's one that gives me a lot of control over how I lay my information out(and an escape plan from).

There are a lot of perspectives on this issue with data silos and walled gardens. But I'm of the opinion that it's a fairly bad state for all of us "end users". Computers to me are about infinite flexibility and malleability, but ironically the tools we have for annotation and remixing are in practice worse than what we have in the physical world. Reading a book in the physical world, I can converse with the author simply by jotting marginalia with my pencil. It's fluid, intuitive, and the medium of paper encourages it(in fact it can't help but be mutated by my use!: pages get bended, stained, torn, etc.). If I want to go further I can add post-it notes to mark interesting passages, I can xerox some pages and create subsections, if it's a magazine I can just tear them all out! That kind of flexibility just isn't available on a computer.

I think it's worth thinking really hard…

If this is interesting to you, you should check out the interesting work that karlicoss and others have done with "Human Programming Interface" [0] / [1].

I've been kicking this idea around for quite a few years and have gone through multiple iterations before finding HPI and tossing out or adapting all my work in favor of building off theirs. Mine is a bit more service / cloud oriented in how it runs (shocker... at one point I made it into a product) while HPI is heavily local-first for obvious good reasons.

HPI is a great platform to build your own stuff off and benefit from all the work that has already been done because imo building a good foundation is the hardest part. Sean Breckenridge's HPI-API is super interesting and useful, could likely be worked into this search engine concept, quite sure Sean actually has both newsboat and Firefox modules already made.

I wrote modules of my own and made an authentication wrapped HPI-API and a GraphQL instance but currently in the middle of an infra move so nothing super cool to show off. At one point I had a dashboard powered by it but we all know how these one-off internal-use-only projects end up. :)

I think the most interesting thing I've written for HPI is my ActivityWatch syncing. It's been happily churning away for years and during my infra move, I discovered that it had about 20GB of activity data stockpiled. Multiple years of down to the second cataloguing of everything I did and saw on the computer.

Lots of…

I was just going to say you should lookup this - https://beepb00p.xyz/myinfra.html - a page I bookmarked and have kept at the back of my head, and turns out it's you!

I have a strong personal need for such a system. I have severe chronic pain that severely disrupts my executive function, and so it's hard to mentally self-direct - so having a visual map of data interconnectivity, where in part I could set view layers based on priority (hide or show different data depending on say if viewing "holistic view" vs. a "project" that's arguably data that's tagged with a project name, etc.

I'm also wanting to write a book on my journal with health and healing, among other things system-society wide that influence health - and allowed my situation to exist and dis-ease progression to occur - and so having a singular system for me to access things like my bookmarks, notes in Notes app, emails filtered based on tags, my comments on HN/Reddit/etc. would really help in allowing me to start to organize and compile different data; perhaps a way to add a non-impacting link between different pieces of data, e.g. say there's a topic in the book related to food and/or water fasting - I could start to interlink all of that content first with links, then another step in the process perhaps with version control integrated - so I can archive a previous/unedited version, keeping the polished/updated version (or multiple data files amalgamated) but linking to their sources/prior versions,…

James is one of the world's great techno-adventurers, & getting to para-socially share in wild adventures like this makes living on Spaceship Earth more lovely & lively! James has also done cool projects like sweet.js macros, helped kick off Firefox devtool's transition to react (iirc), oh and lead the basically industry standard JS formatter Priettier project. I'm forgetting a dozen other things over the years but it's always been fun.

Just a heads-up, the File System Access API[1] is underway in Chrome, which potentially removes nearly all of the absurdity here. It has other benefits too. A web page using this could write a .sql file on to your drive, that other programs could then access. One of the other bright stars in my world is Karli Koss, who has an extensive personal data-extraction setup for a ridiculously colossal variety of services & devices[2]. A vast amount of this massive massive data-gathering framework is just reading sqlite databases of the various devices and apps. If the web can help participate more actively, can let apps write sql files to store state: so much the better I say. Help externalize your state beyond the browser, please!

[1] https://wicg.github.io/file-system-access/#api-filesystemwritablefilestream https://caniuse.com/native-filesystem-api

[2] https://beepb00p.xyz/myinfra.html