From 9e22a04bf4ee0b2d36ae670e623521297d0e1a2d Mon Sep 17 00:00:00 2001 From: mj-saunders Date: Thu, 1 May 2025 19:15:00 +0000 Subject: [PATCH] Update Developer-Path_Implementing-Commons-Using-ActivityPub-Groups-and-Hashtags --- ...g-ActivityPub%2C-Groups%2C-and-Hashtags.md | 203 --------------- ...Using-ActivityPub-Groups-and-Hashtags.-.md | 232 ++++++++++++++++++ 2 files changed, 232 insertions(+), 203 deletions(-) delete mode 100644 Developer-Path%3A-Implementing-Commons-Using-ActivityPub%2C-Groups%2C-and-Hashtags.md create mode 100644 Developer-Path_Implementing-Commons-Using-ActivityPub-Groups-and-Hashtags.-.md diff --git a/Developer-Path%3A-Implementing-Commons-Using-ActivityPub%2C-Groups%2C-and-Hashtags.md b/Developer-Path%3A-Implementing-Commons-Using-ActivityPub%2C-Groups%2C-and-Hashtags.md deleted file mode 100644 index 11442a7..0000000 --- a/Developer-Path%3A-Implementing-Commons-Using-ActivityPub%2C-Groups%2C-and-Hashtags.md +++ /dev/null @@ -1,203 +0,0 @@ -Developer Path: Implementing Commons Using ActivityPub, Groups, and Hashtags - -We’re exploring how to use ActivityPub for federated, editable content in a "commons" structure. The aim is for data objects to be: - - Pushed by a user account, - - Placed in the commons, - - Editable by trusted groups, - - Organized by hashtags, - - Federated across instances. - -This resembles a wiki-like system within ActivityPub, but with trust-grouped, hashtag-driven object flows. Groups as Commons? We likely don’t need traditional “groups” in the Facebook sense. Instead, collections of hashtags, and logic based on hashtags can act as implicit groups. - -Each object is still owned by a user? (for federation reasons?), but logically "shared" with others via inclusion in a "commons group". Q: Can a “group” just be a collection of hashtags with boolean logic applied? If so, the group becomes an object itself—recursively building a metadata mesh. - -Q: Collections & storage? Collections are part of the core ActivityPub spec, but vague in meaning. Some are ordered (e.g., outbox, followers). Others are unordered or conceptual. Collections seem to be API-level constructs, not strict data models. -(Ben): ActivityPub is just the "protocol" - so Collections are just how records are represented in places like an actor's outbox. Most implementations use `OrderedCollection`s for just about everything, with un-ordered collections just being left unused in the spec. In Emissary, these records are stored in MongoDB. These map (nearly) 1:1 with ActivityPub collections, but there are differences, such as the extra data that Emissary stores but does not represent via ActivityPub. - -Q: Is this table structure scalable for thousands of hashtags/groups? The heavy lifting of flows can be handled by back-end logic. -(Ben): I believe so. This specific code hasn't been tested with more than a few thousand records in each MongoDB table. But, MongoDB is built for high-capacity and high-volume datasets. I think we'll be in good hands here. - -Federation logic & edits, object ownership still belongs to the originating account/instance. Others in the trust group can edit via the ActivityPub Update verb. This is different from the usual Comment/Reply flow. The system needs history/versioning of objects like a wiki. -(Ben): Agreed. Emissary doesn't currently support this kind of workflow, but I'd like to see how possible it is. I have reached out to the developer behind Ibis Wiki to see if he has any suggestions on how we can accomplish this, too. - -Q: How does current Fediverse federation handle Update? Does it propagate changes cleanly? What do Lemmy, Peertube, etc. do here? -(Ben): Every system works a little differently. There are some common data elements that are nearly universal, but then each system has its own extensions. It's hit-and-miss for many kinds of interop. Check out [Fediverse Enhancement Proposals](https://codeberg.org/fediverse/fep) for some examples of the different extensions that everyone is supporting. Since each app has a different target and feature set, the question is "what do you want to sync to Lemmy/Peertube users?" -- probably just *the fact* that this record exists (or has changed) but not the actual changes themselves. A Peertube user can't edit a Wiki article from Peertube itself, they'll need to bounce over to the wiki to do it. - -Metadata & hashtag logic, hashtags act as the primary organizing mechanism. We avoid enforcing a single tag; instead, instances can alias/group tags (e.g. treating #openweb, #4opens, #fediverse as equivalent in one context). This can be done by injecting additional hashtags via trust-logic (i.e., metadata layering). Injected hashtags must not appear as user-authored. Display should abstract hashtags in the UI - show clean flows, not cluttered tags. -(Ben): I'm interested to explore this with you. I've built some interesting Hashtag filters for Bandwagon. In short, users can hashtag anything they want, but server admins choose which ones will work in the global search, and which ones get banned. I can walk you through this, if it's helpful. And, if you have more sophisticated ideas on managing hashtags better, I'm happy to incorporate this into Emissary itself. - -Q: Do we need a better metadata field than hashtags? Or can we repurpose hashtags in a way that supports this abstraction? -(Ben): Emissary has rudimentary support for different "flavors" of hashtags -- though I haven't really used this to its full potential yet. As an example, Bandwagon.fm uses different flavors of hashtags to represent: Genres (rock, pop, jazz, etc), Moods (cozy, party, sleepy, etc) and Instrumentation (Guitar, Piano, Violin, etc). Perhaps there's a way for us to use this hashtag hierarchy in the Wiki project as well. - -Trust & authorization flows, only people in a trusted group can edit an object in the commons. Others can view/comment but not edit. Trust flows are likely to diverge into subjective consensus spaces - and that’s fine. -(Ben): This is good. Emissary has some strong user permission controls in place now, though this may be difficult to manage if we allow distributed edits. One thought here: this might work if ActivityPub is used to push COPIES of an article to external servers, but EDITS/UPDATES to the original must always be done on the original server. As in Git, external servers might choose to FORK a piece of content, but would then no longer get updates from the original. Just a thought. - -Q: How is authorization defined and verified across instances? -(Ben): We could use OAuth for this. Emissary has nascent support for OAuth, and we could expand this to allow someone to verify themselves via their home server. - -Templating & UX abstraction, hashtag chaos needs to be hidden (mediated) in the UI with templated displays. Instances could share templates for rendering object streams. We’ll need a UX spec for how to display, merge, or alias tags and flows. - -Q: Can templates be federated or instance-local? What’s the minimum needed to keep UX in sync? -(Ben): Templates are under the control of the server admin, and must be installed on each instance. For two instances to Federate the same kind of content, they should probably install the same sets of templates. Templates can be published via Git, and other common file systems, but there's no automated way to push them to other servers (and it would probably be a security risk if we tried to allow it) - -Scaling Considerations: - -Everything becomes a data object: posts, groups, tags, users. - -Each has their own flows, metadata, and permissions. - -It gets meta and messy, but ActivityPub supports it in theory. - -We need to measure early performance scaling under large flows and tag aliasing. - -Q: What flows (hashtags, groups, trust edits) are most likely to create bottlenecks? -(Ben): The biggest bottlenecks are likely to be interoperability with other apps. It's "easy" to interop with other servers running your own software. But it's hard to make messages that are recognized by all the crazy permutations out there on the Fediverse. - -Developer Questions: - -How are you currently handling group logic? Do you use ActivityPub collections or something custom? - -How do you store and retrieve group-object relationships in the database? - -Are hashtags treated as first-class group indicators? - -How does your system manage Update verbs and object versioning? - -Do we have support for metadata (like added hashtags) in a way that’s distinct from user-generated content? - -How is authorization for edits determined and verified? - -Can we understand how the system would scale with thousands of hashtag-defined groups and flows? - -Need a session for mical to understand the template system for UX rendering of objects or flows? - -Next Steps - -Define minimal data schema: objects, groups, tags, flows, users. - -Prototype Update logic with versioning. - -Test tag-aliasing with injected metadata. - -Write draft UX spec for hashtag/flow abstraction. - -Implement basic trust-group logic using existing ActivityPub verbs. - ----------------------- - -DRAFT - OMN Commons System — Developer Roadmap -Phase 1: Foundations - -Goals: Understand current Fediverse norms, pick a viable path, lay groundwork. - -* Review existing ActivityPub specs - - Collections, Groups (FEP-1b12), Update/Delete flows, Object types. - -* Decide on a Grouping Approach - - Evaluate Lemmy/Peertube models - - Choose between implementing FEP-1b12 or a more custom logic using hashtags - -* Phase 2: do a working version using Emissary - - -* Phase 3: Core Object Model - -Goals: Build basic ActivityPub-compliant object handling. - -Implement UPDATE as primary user interaction - -Implement object history/versioning (wiki-style) - -Phase 4: Federation - -Goals: Federate content using ActivityPub, handle common verbs. - -Outbox/Inbox endpoint logic - -Signature validation and auth - -Object ID resolution and deduplication - -Propagation of edits (trust-based or rules-based) - -Support federation of UPDATE alongside Create - -Phase 5: Group + Commons Logic - -Goals: Implement logic for groups, commons access, and authorization. - -Group object model (as virtual objects or ActivityPub actors) - -Implement group membership flows - -Store group-object relationships in DB? - -Boolean logic for hashtag-group relations - -UX: Display objects by hashtag flow + group context - -Define “Commons” group as a writable shared space - -Phase 6: Trust + Authorization - -Goals: Mediate access to objects based on trust logic. - -Define and store "trust flow" data (user-group relationships) - -Build auth logic for edit vs. comment rights - -Implement permissions at the object and group level - -UX feedback for permission errors - -Phase 7: Scaling + UX Flows - -Goals: Refine UX and scale up the logic. - -Metadata injection via instance logic (e.g. grouping hashtags) - -Display hashtags abstractly (UX templating of messy tags) - -Optimize object retrieval (object IDs by group → fetch only needed) - -Caching and pagination for large flows - -Tag aliasing/grouping UI tools - -Phase 8: Application Launch - -Goals: Polish and release a working prototype instance. - -“MakingHistory” and “Indymedia” themed templates - -MVP instance of federated “commons” publishing - -Simple mobile-first UI with edit/comment modes - -Editable content histories - -Federation with at least 1-2 existing Fediverse projects - -**Developer Questions (Ongoing)** - - How does our chosen backend implement DB relationships? - - Can we implement "groups" using only hashtags and aliasing logic? - - How do we scale boolean logic across thousands of hashtags? - - What’s the best way to model UPDATE as primary interaction in ActivityPub? - - Can we cache and compress federated flows without breaking expected behaviors? - - How to handle clashing edits on shared (commons) objects? - - What are the best existing tools for ActivityPub testing/debugging? \ No newline at end of file diff --git a/Developer-Path_Implementing-Commons-Using-ActivityPub-Groups-and-Hashtags.-.md b/Developer-Path_Implementing-Commons-Using-ActivityPub-Groups-and-Hashtags.-.md new file mode 100644 index 0000000..8a22640 --- /dev/null +++ b/Developer-Path_Implementing-Commons-Using-ActivityPub-Groups-and-Hashtags.-.md @@ -0,0 +1,232 @@ +## Developer Path: Implementing Commons Using ActivityPub, Groups, and Hashtags + +We’re exploring how to use ActivityPub for federated, editable content in a "commons" structure. The aim is for data objects to be: + +- Pushed by a user account +- Placed in the commons +- Editable by members of trusted groups +- Organized by hashtags +- Federated across instances + +This resembles a wiki-like system within ActivityPub, but with trust-grouped, hashtag-driven object flows. + +### Questions + +#### Q. Can Groups be used as a Commons? + +We likely don’t need traditional “groups” in the Facebook sense. Instead, collections of hashtags, and logic based on hashtags can act as implicit groups. + +> [saunders]: I feel like this might be confusing. Do we not need to define "group" with regard to members that come from one or more instances. +> Also we should acknowledge that we're almost certainly using "group" in a different sense to what other fediverse implementations are working on - do we want to find a different term, or try and broaden an existing one? + +#### Q. Is each object still owned by a user? (for federation reasons?) + +...but logically "shared" with others via inclusion in a "commons group". + + +#### Q. Can a “group” just be a collection of hashtags with boolean logic applied? + +If so, the group becomes an object itself - recursively building a metadata mesh. + +> [saunders]: I'm not sure how a group as an object might look within ActivityPub - need to read the specs some more. Especially given that I presume any given groups definition would be somewhat fluid - adding/removing hashtags over time. While I feel like it makes sense to me as some evolving metadata, I again find the term "group" a little confusing in the context. +> When talking about a single "object" owned by a single user, what is an example object in this context? A whole document (in the same way a single toot is on mastodon). If so, I think we might want to further discuss the idea of a message being an "edit" (effectively a git diff/commit); i.e. a change to a document (object). The first edit would simply be the difference between an empty object and the initial content. +> Then much like a multitude of replies to a single toot in mastodon can be displayed as a thread in the UI, we could (re)construct a single document to display based on the thread of diffs. +> Am I getting too deep into the weeds too early? + +#### Q. Collections & storage? + +Collections are part of the core ActivityPub spec, but vague in meaning. +Some are ordered (e.g., outbox, followers). Others are unordered or conceptual. +Collections seem to be API-level constructs, not strict data models. + +#### Q. Is this table structure scalable for thousands of hashtags/groups? + +The heavy lifting of flows can be handled by back-end logic. + +Federation logic & edits, object ownership still belongs to the originating account/instance. Others in the trust group can edit via the ActivityPub Update verb. This is different from the usual Comment/Reply flow. The system needs history/versioning of objects like a wiki. + +#### Q: How does current Fediverse federation handle Update? + +Does it propagate changes cleanly? What do Lemmy, Peertube, etc. do here? + +Metadata & hashtag logic, hashtags act as the primary organizing mechanism. We avoid enforcing a single tag; instead, instances can alias/group tags (e.g. treating #openweb, #4opens, #fediverse as equivalent in one context). This can be done by injecting additional hashtags via trust-logic (i.e., metadata layering). Injected hashtags must not appear as user-authored. Display should abstract hashtags in the UI - show clean flows, not cluttered tags. + +> [saunders]: Not sure how to go about this. Injecting hashtags feels a bit off. They would have to be somewhat separate from normal hashtag use, but then other fediverse instances would not recognise and them as normal hashtags... +> But if they're intended only for "internal" use and not really for more convential use by e.g. a mastodon instance, then I think it would be fine to store them separately. + +#### Q: Do we need a better metadata field than hashtags? Or can we repurpose hashtags in a way that supports this abstraction? + +> [saunders]: Going on my comment above, I feel like maybe we just use a separate field. + +Trust & authorization flows, only people in a trusted group can edit an object in the commons. Others can view/comment but not edit. Trust flows are likely to diverge into subjective consensus spaces - and that’s fine. + +> [saunders]: We need to clear up the ambiguity between group as a set of people vs group as something determined by a set of hashtags. + +#### Q: How is authorization defined and verified across instances? + +Templating & UX abstraction, hashtag chaos needs to be hidden (mediated) in the UI with templated displays. Instances could share templates for rendering object streams. We’ll need a UX spec for how to display, merge, or alias tags and flows. + +> [saunders]: Feels like another reason to use a separate data field, and not whatever is currently used for hashtags. That said, we may still make use of normal hashtags for some filtering, as they usually are. + +#### Q: Can templates be federated or instance-local? + +> [saunders]: "Template" in the `emmisary` context? Or other? +> Federated in what sense? + +What’s the minimum needed to keep UX in sync? + +> [saunders]: If I'm understanding right, I get the impression it would require some amount of shared codebase. +> If you're proposing that a template exist as a shared object within the Commons, that could get complicated; though perhaps not impossible. + + +### Scaling Considerations + +Everything becomes a data object: posts, groups, tags, users. + +Each has their own flows, metadata, and permissions. + +It gets meta and messy, but ActivityPub supports it in theory. + +We need to measure early performance scaling under large flows and tag aliasing. + +> [saunders]: I don't yet know enough about ActivityPub to chime in meaningfully on this. + +**Q:** What flows (hashtags, groups, trust edits) are most likely to create bottlenecks? + + +### Developer Questions + +- How are you currently handling group logic? Do you use ActivityPub collections or something custom? +- How do you store and retrieve group-object relationships in the database? +- Are hashtags treated as first-class group indicators? +- How does your system manage Update verbs and object versioning? +- Do we have support for metadata (like added hashtags) in a way that’s distinct from user-generated content? +- How is authorization for edits determined and verified? +- Can we understand how the system would scale with thousands of hashtag-defined groups and flows? +- Need a session for mical to understand the template system for UX rendering of objects or flows? + + +### Next Steps + +- Define minimal data schema: objects, groups, tags, flows, users. +- Prototype Update logic with versioning. +- Test tag-aliasing with injected metadata. +- Write draft UX spec for hashtag/flow abstraction. +- Implement basic trust-group logic using existing ActivityPub verbs. + + +---------------------- + + +DRAFT - OMN Commons System — Developer Roadmap + +Phase 1: Foundations + +Goals: Understand current Fediverse norms, pick a viable path, lay groundwork. + +* Review existing ActivityPub specs + + Collections, Groups (FEP-1b12), Update/Delete flows, Object types. + +* Decide on a Grouping Approach + + Evaluate Lemmy/Peertube models + + Choose between implementing FEP-1b12 or a more custom logic using hashtags + +* Phase 2: do a working version using Emissary + + +* Phase 3: Core Object Model + +Goals: Build basic ActivityPub-compliant object handling. + +Implement UPDATE as primary user interaction + +Implement object history/versioning (wiki-style) + +Phase 4: Federation + +Goals: Federate content using ActivityPub, handle common verbs. + +Outbox/Inbox endpoint logic + +Signature validation and auth + +Object ID resolution and deduplication + +Propagation of edits (trust-based or rules-based) + +Support federation of UPDATE alongside Create + +Phase 5: Group + Commons Logic + +Goals: Implement logic for groups, commons access, and authorization. + +Group object model (as virtual objects or ActivityPub actors) + +Implement group membership flows + +Store group-object relationships in DB? + +Boolean logic for hashtag-group relations + +UX: Display objects by hashtag flow + group context + +Define “Commons” group as a writable shared space + +Phase 6: Trust + Authorization + +Goals: Mediate access to objects based on trust logic. + +Define and store "trust flow" data (user-group relationships) + +Build auth logic for edit vs. comment rights + +Implement permissions at the object and group level + +UX feedback for permission errors + +Phase 7: Scaling + UX Flows + +Goals: Refine UX and scale up the logic. + +Metadata injection via instance logic (e.g. grouping hashtags) + +Display hashtags abstractly (UX templating of messy tags) + +Optimize object retrieval (object IDs by group → fetch only needed) + +Caching and pagination for large flows + +Tag aliasing/grouping UI tools + +Phase 8: Application Launch + +Goals: Polish and release a working prototype instance. + +“MakingHistory” and “Indymedia” themed templates + +MVP instance of federated “commons” publishing + +Simple mobile-first UI with edit/comment modes + +Editable content histories + +Federation with at least 1-2 existing Fediverse projects + +**Developer Questions (Ongoing)** + + How does our chosen backend implement DB relationships? + + Can we implement "groups" using only hashtags and aliasing logic? + + How do we scale boolean logic across thousands of hashtags? + + What’s the best way to model UPDATE as primary interaction in ActivityPub? + + Can we cache and compress federated flows without breaking expected behaviors? + + How to handle clashing edits on shared (commons) objects? + + What are the best existing tools for ActivityPub testing/debugging? \ No newline at end of file