MineContext: a ByteDance product that lets AI understand what you do all day**
GitHub: https://github.com/volcengine/MineContext/tree/main
When you chat with or ask questions to ChatGPT, you’re usually limited by prompt length and time cost. You can only provide a bit of context at a time, and there’s no way for AI to truly understand what we’re working on.
The problem MineContext wants to solve is exactly this: “AI can’t remember your world.”
It’s an open-source context-aware AI partner that understands your digital world through screenshots and content parsing — whether you’re writing code, reading docs, looking things up, or sitting in a meeting, it can automatically recognize and extract information. Based on that context, the AI generates daily reports, summaries, to-dos, and insights, and proactively pushes them to you, so you never have to explain the background over and over again.
All data is stored locally, and it supports models like OpenAI and Doubao (you’ll need to use your own API).
The name MineContext is a clever one: it means both “my context” and “mine context.” It was inspired by Minecraft — if the vast amount of context is “blocks” scattered all over the place, then MineContext is the “world” that lets you freely build, combine, and create anew.
Documentation: How to get started with MineContext quickly
At first glance, this thing seems pretty similar to the DayFlow I used in September last year.
I could give it another try. I stopped using DayFlow back then because it used local compute for video analysis, and the analysis was way too inaccurate. I was using that Qwen 4B vision model, and the analysis was so inaccurate that I just dropped it.
I just took a look at the official user manual, and its features are exactly the same as DayFlow’s.
But its visual understanding requires a Volcano Engine API key. I saw people in the group saying it costs about one yuan a day — that’s just absurd.
If my visual understanding works by uploading screenshots online, waiting for them to be processed, and then having the labels sent back to me locally, isn’t that still the same privacy leak?
My local screen is being shown to others every single day, and on top of that they still charge me?
Products like this still have to rely on visual understanding models. They’ll only really take off once everyone can deploy them locally and enterprises can run them at scale too.
This current stage of development is a bit like early computers:
- The client machine’s specs definitely can’t be very high.
- As a terminal, it has to connect to a remote mainframe and use the mainframe’s compute to run its calculations.
But after computers had developed for a few years, households everywhere could buy a Mac or an IBM machine, and all the features were already there. All those services and features that once required connecting a terminal to a remote system were now implemented independently in the local client, and could run with no internet connection at all.
Mapping that onto today’s AI era, I think it’s equivalent to the early days when terminals were still in use.
The tokens we buy now all need the cloud to process them for us, like paying an electricity bill. But this won’t last long — in ten years at most, we’ll go from the terminal era to the home computer era, and all tokens will run locally.