indymedia
/
epicyon
mirror of https://gitlab.com/bashrc2/epicyon
7
4
Fork 1
Code Issues 14 Releases Wiki Activity

Merge branch 'main' of ssh://code.freedombone.net:2222/bashrc/epicyon into main

main
Bob Mottram 2021-03-22 18:28:11 +00:00
parent fb1cd34eef 34f0cb47ad
commit b71f1c93cb
28 changed files with 3821 additions and 1870 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -8,7 +8,7 @@ Add issues on https://gitlab.com/bashrc2/epicyon/-/issues
<img src="https://epicyon.net/img/mobile.jpg" width="30%"/>
Epicyon is a modern [ActivityPub](https://www.w3.org/TR/activitypub) compliant server implementing both S2S and C2S protocols and sutable for installation on single board computers. It includes features such as moderation tools, post expiry, content warnings, image descriptions, news feed and perimeter defense against adversaries. It contains *no javascript* and uses HTML+CSS with a Python backend.
Epicyon is a modern [ActivityPub](https://www.w3.org/TR/activitypub) compliant server implementing both S2S and C2S protocols and suitable for installation on single board computers. It includes features such as moderation tools, post expiry, content warnings, image descriptions, news feed and perimeter defense against adversaries. It contains *no JavaScript* and uses HTML+CSS with a Python backend.
[Project Goals](README_goals.md) - [Commandline interface](README_commandline.md) - [Customizations](README_customizations.md) - [Code of Conduct](code-of-conduct.md)
@ -238,7 +238,7 @@ Please be aware that such installations will not federate with ordinary fedivers
## Custom Fonts
If you want to use a particular font then copy it into the *fonts* directory, rename it as *custom.ttf/woff/woff2/otf* and then restart the epicyon daemon.
If you want to use a particular font then copy it into the *fonts* directory, rename it as *custom.ttf/woff/woff2/otf* and then restart the Epicyon daemon.
``` bash
systemctl restart epicyon

63
README_commandline.md
View File

@ -1,6 +1,6 @@
# Commandline Admin
# Command-line Admin
This system can be administrated from the commandline.
This system can be administrated from the command-line.
## Account Management
@ -52,7 +52,7 @@ To remove an account (be careful!):
python3 epicyon.py --rmgroup nickname@domain
```
Setting avatar or changing background is the same as for any other account on the system. You can also moderate a group, applying filters, blocks or a perimeter, in the same way as for other acounts.
Setting avatar or changing background is the same as for any other account on the system. You can also moderate a group, applying filters, blocks or a perimeter, in the same way as for other accounts.
## Defining a perimeter
@ -76,7 +76,7 @@ The password is for the client to obtain access to the server.
You may or may not need to use the *--port*, *--https* and *--tor* options, depending upon how your server was set up.
Unfollowing is silimar:
Unfollowing is similar:
``` bash
python3 epicyon.py --nickname [yournick] --domain [name] --unfollow othernick@domain --password [c2s password]
@ -131,12 +131,22 @@ To view the public posts for a person:
python3 epicyon.py --posts nickname@domain
```
If you want to view the raw json:
If you want to view the raw JSON:
``` bash
python3 epicyon.py --postsraw nickname@domain
```
## Getting the JSON for your timelines
The **--posts** option applies for any ActivityPub compatible fediverse account with visible public posts. You can also use an authenticated version to obtain the paginated JSON for your inbox, outbox, direct messages, etc.
``` bash
python3 epicyon.py --nickname [yournick] --domain [yourdomain] --box [inbox|outbox|dm] --page [number] --password [yourpassword]
```
You could use this to make your own c2s client, or create your own notification system.
## Listing referenced domains
To list the domains referenced in public posts:
@ -156,7 +166,7 @@ xdot socnet.dot
## Delete posts
To delete a post which you wrote you must first know its url. It is usually something like:
To delete a post which you wrote you must first know its URL. It is usually something like:
``` text
https://yourDomain/users/yourNickname/statuses/number
@ -177,7 +187,7 @@ Another complication of federated deletion is that the followers collection may
## Announcements/repeats/boosts
To announce or repeat a post you will first need to know it's url. It is usually something like:
To announce or repeat a post you will first need to know it's URL. It is usually something like:
``` text
https://domain/users/name/statuses/number
@ -192,7 +202,7 @@ python3 epicyon.py --nickname [yournick] --domain [name] \
## Like posts
To like a post you will first need to know it's url. It is usually something like:
To like a post you will first need to know it's URL. It is usually something like:
``` text
https://domain/users/name/statuses/number
@ -240,7 +250,7 @@ Whether you are using the **--federate** option to define a set of allowed insta
python3 epicyon.py --nickname yournick --domain yourdomain --block somenick@somedomain --password [c2s password]
```
This blocks at the earliest possble stage of receiving messages, such that nothing from the specified account will be written to your inbox.
This blocks at the earliest possible stage of receiving messages, such that nothing from the specified account will be written to your inbox.
Or to unblock:
@ -248,6 +258,22 @@ Or to unblock:
python3 epicyon.py --nickname yournick --domain yourdomain --unblock somenick@somedomain --password [c2s password]
```
## Bookmarking
You may want to bookmark posts for later viewing or replying. This can be done via c2s with the following:
``` bash
python3 epicyon.py --nickname yournick --domain yourdomain --bookmark [post URL] --password [c2s password]
```
Note that the URL must be that of an ActivityPub post in your timeline. Any other URL will be ignored.
And to undo the bookmark:
``` bash
python3 epicyon.py --nickname yournick --domain yourdomain --unbookmark [post URL] --password [c2s password]
```
## Filtering on words or phrases
Blocking based upon the content of a message containing certain words or phrases is relatively crude and not always effective, but can help to reduce unwanted communications.
@ -313,7 +339,7 @@ python3 epicyon.py --nickname [admin nickname] --domain [mydomain] \
--password [c2s password]
```
This extends the ActivityPub client-to-server protocol to include activities called *Delegate* and *Role*. The json looks like:
This extends the ActivityPub client-to-server protocol to include activities called *Delegate* and *Role*. The JSON looks like:
``` json
{ 'type': 'Delegate',
@ -343,7 +369,7 @@ python3 epicyon.py --nickname [nick] --domain [mydomain] \
The level value is a percentage which indicates how proficient you are with that skill.
This extends the ActivityPub client-to-server protocol to include an activity called *Skill*. The json looks like:
This extends the ActivityPub client-to-server protocol to include an activity called *Skill*. The JSON looks like:
``` json
{ 'type': 'Skill',
@ -365,7 +391,7 @@ python3 epicyon.py --nickname [nick] --domain [mydomain] \
The status value can be any string, and can become part of organization building by combining it with roles and skills.
This extends the ActivityPub client-to-server protocol to include an activity called *Availability*. "Status" was avoided because of te possibility of confusion with other things. The json looks like:
This extends the ActivityPub client-to-server protocol to include an activity called *Availability*. "Status" was avoided because of the possibility of confusion with other things. The JSON looks like:
``` json
{ 'type': 'Availability',
@ -377,7 +403,7 @@ This extends the ActivityPub client-to-server protocol to include an activity ca
## Shares
This system includes a feature for bartering or gifting (i.e. common resource pooling or exchange without money), based upon the earlier Sharings plugin made by the Las Indias group which existed within GNU Social. It's intended to operate at the municipal level, sharing physical objects with people in your local vicinity. For example, sharing gardening tools on a street or a 3D printer between makerspaces.
This system includes a feature for bartering or gifting (i.e. common resource pooling or exchange without money), based upon the earlier Sharings plugin made by the Las Indias group which existed within GNU Social. It's intended to operate at the municipal level, sharing physical objects with people in your local vicinity. For example, sharing gardening tools on a street or a 3D printer between maker-spaces.
To share an item.
@ -385,7 +411,7 @@ To share an item.
python3 epicyon.py --itemName "spanner" --nickname [yournick] --domain [yourdomain] --summary "It's a spanner" --itemType "tool" --itemCategory "mechanical" --location [yourCity] --duration "2 months" --itemImage spanner.png --password [c2s password]
```
For the duration of the share you can use hours,days,weeks,months or years.
For the duration of the share you can use hours, days, weeks, months, or years.
To remove a shared item:
@ -422,7 +448,7 @@ Or if you have picospeaker installed:
The desktop client has a few commands, which may be more convenient than the web interface for some purposes:
``` bash
quit Exit from the notification client
quit Exit from the desktop client
mute Turn off the screen reader
speak Turn on the screen reader
sounds on Turn on notification sounds
@ -430,17 +456,22 @@ sounds off Turn off notification sounds
rp Repeat the last post
like Like the last post
unlike Unlike the last post
bookmark Bookmark the last post
unbookmark Unbookmark the last post
mute Mute the last post
unmute Unmute the last post
reply Reply to the last post
post Create a new post
post to [handle] Create a new direct message
announce/boost Boost the last post
follow [handle] Make a follow request
unfollow [handle] Stop following the give handle
show dm|sent|inbox|replies Show a timeline
show dm|sent|inbox|replies|bookmarks Show a timeline
next Next page in the timeline
prev Previous page in the timeline
read [post number] Read a post from a timeline
open [post number] Open web links within a timeline post
profile [post number] Show profile for the person who made the given post
```
If you have a GPG key configured on your local system and are sending a direct message to someone who has a PGP key (the exported key, not just the key ID) set as a tag on their profile then it will try to encrypt the message automatically. So under some conditions end-to-end encryption is possible, such that the instance server only sees ciphertext. Conversely, for arriving direct messages if they are PGP encrypted then the desktop client will try to obtain the relevant public key and decrypt.

2
README_customizations.md
View File

@ -28,4 +28,4 @@ Extra emoji can be added to the *emoji* directory and you should then update the
## Themes
If you want to create a new theme then the functions for that are within *theme.py*. These functions take the css templates and modify them. You will need to edit *themesDropdown* within *webinterface.py* and add the appropriate translations for the theme name. Themes are selectable from the profile screen of the administrator.
If you want to create a new theme then the functions for that are within *theme.py*. These functions take the CSS templates and modify them. You will need to edit *themesDropdown* within *webinterface.py* and add the appropriate translations for the theme name. Themes are selectable from the profile screen of the administrator.

18
README_goals.md
View File

@ -10,22 +10,22 @@
* Attention to accessibility and should be usable in lynx with a screen reader
* Remove metadata from attached images, avatars and backgrounds
* Support for multiple themes, with ability to create custom themes
* Being able to build crowdsouced organizations with roles and skills
* Being able to build crowd-sourced organizations with roles and skills
* Sharings collection, similar to the gnusocial sharings plugin
* Quotas for received posts per day, per domain and per account
* Hellthread detection and removal
* Hell-thread detection and removal
* Instance and account level federation lists
* Support content warnings, reporting and blocking
* http signatures and basic auth
* json-LD signatures on outgoing posts, optional on incoming
* Compatible with http (onion addresses, i2p), https and hypercore
* JSON-LD signatures on outgoing posts, optional on incoming
* Compatible with HTTP (onion addresses, i2p), HTTPS and hypercore
* Minimal dependencies
* Dependencies are maintained Debian packages
* Data minimization principle. Configurable post expiry time
* Likes and repeats only visible to authorized viewers
* ReplyGuy mitigation - maxmimum replies per post or posts per day
* Reply Guy mitigation - maximum replies per post or posts per day
* Ability to delete or hide specific conversation threads
* Commandline interface
* Command-line interface
* Simple web interface
* Designed for intermittent connectivity. Assume network disruptions
* Limited visibility of follows/followers
@ -36,17 +36,17 @@
**Features which won't be implemented**
The following are considered antifeatures of other social network systems, since they encourage dysfunctional social interactions.
The following are considered anti-features of other social network systems, since they encourage dysfunctional social interactions.
* Features designed to scale to large numbers of accounts (say, more than 20 active users)
* Trending hashtags, or trending anything
* Ranking, rating or recommending mechanisms for posts or people (other than likes or repeats/boosts)
* Geolocation features
* Geo-location features
* Algorithmic timelines (i.e. non-chronological)
* Direct payment mechanisms, although integration with other services may be possible
* Any variety of blockchain
* Sponsored posts
* Enterprise features for use cases applicable only to businesses. Epicyon could be used in a small business, but it's not primarily designed for that
* Collaborative editing of posts, although you could do that outside of this system using etherpad, or similar
* Collaborative editing of posts, although you could do that outside of this system using Etherpad, or similar
* Anonymous posts from random internet users published under a single generic instance account
* Hierarchies of roles beyond ordinary moderation, such as X requires special agreement from Y before sending a post

92
announce.py
View File

@ -225,8 +225,8 @@ def sendAnnounceViaServer(baseDir: str, session,
print('DEBUG: announce webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: announce webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -242,11 +242,12 @@ def sendAnnounceViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: announce no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: announce no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -257,11 +258,90 @@ def sendAnnounceViaServer(baseDir: str, session,
'Authorization': authHeader
}
postResult = postJson(session, newAnnounceJson, [], inboxUrl,
headers, 30, True)
headers, 3, True)
if not postResult:
print('WARN: Announce not posted')
print('WARN: announce not posted')
if debug:
print('DEBUG: c2s POST announce success')
return newAnnounceJson
def sendUndoAnnounceViaServer(baseDir: str, session,
undoPostJsonObject: {},
nickname: str, password: str,
domain: str, port: int,
httpPrefix: str, repeatObjectUrl: str,
cachedWebfingers: {}, personCache: {},
debug: bool, projectVersion: str) -> {}:
"""Undo an announce message via c2s
"""
if not session:
print('WARN: No session for sendUndoAnnounceViaServer')
return 6
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
handle = actor.replace('/users/', '/@')
statusNumber, published = getStatusNumber()
unAnnounceJson = {
'@context': 'https://www.w3.org/ns/activitystreams',
'id': actor + '/statuses/' + str(statusNumber) + '/undo',
'type': 'Undo',
'actor': actor,
'object': undoPostJsonObject['object']
}
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: undo announce webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: undo announce webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
# get the actor inbox for the To handle
(inboxUrl, pubKeyId, pubKey, fromPersonId,
sharedInbox, avatarUrl,
displayName) = getPersonBox(baseDir, session, wfRequest,
personCache,
projectVersion, httpPrefix,
nickname, domain,
postToBox, 73528)
if not inboxUrl:
if debug:
print('DEBUG: undo announce no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: undo announce no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(nickname, password)
headers = {
'host': domain,
'Content-type': 'application/json',
'Authorization': authHeader
}
postResult = postJson(session, unAnnounceJson, [], inboxUrl,
headers, 3, True)
if not postResult:
print('WARN: undo announce not posted')
if debug:
print('DEBUG: c2s POST undo announce success')
return unAnnounceJson

13
availability.py
View File

@ -108,11 +108,11 @@ def sendAvailabilityViaServer(baseDir: str, session,
domain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: availability webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: availability webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -127,11 +127,12 @@ def sendAvailabilityViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: availability no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: availability no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(nickname, password)
@ -144,7 +145,7 @@ def sendAvailabilityViaServer(baseDir: str, session,
postResult = postJson(session, newAvailabilityJson, [],
inboxUrl, headers, 30, True)
if not postResult:
print('WARN: failed to post availability')
print('WARN: availability failed to post')
if debug:
print('DEBUG: c2s POST availability success')

266
blocking.py
View File

@ -7,7 +7,11 @@ __email__ = "bob@freedombone.net"
__status__ = "Production"
import os
import json
from datetime import datetime
from utils import getCachedPostFilename
from utils import loadJson
from utils import saveJson
from utils import fileLastModified
from utils import setConfigParam
from utils import hasUsersPath
@ -361,6 +365,268 @@ def outboxUndoBlock(baseDir: str, httpPrefix: str,
print('DEBUG: post undo blocked via c2s - ' + postFilename)
def mutePost(baseDir: str, nickname: str, domain: str, port: int,
httpPrefix: str, postId: str, recentPostsCache: {},
debug: bool) -> None:
""" Mutes the given post
"""
postFilename = locatePost(baseDir, nickname, domain, postId)
if not postFilename:
return
postJsonObject = loadJson(postFilename)
if not postJsonObject:
return
if postJsonObject.get('object'):
if isinstance(postJsonObject['object'], dict):
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
# does this post have ignores on it from differenent actors?
if not postJsonObject['object'].get('ignores'):
if debug:
print('DEBUG: Adding initial mute to ' + postId)
ignoresJson = {
"@context": "https://www.w3.org/ns/activitystreams",
'id': postId,
'type': 'Collection',
"totalItems": 1,
'items': [{
'type': 'Ignore',
'actor': actor
}]
}
postJsonObject['object']['ignores'] = ignoresJson
else:
if not postJsonObject['object']['ignores'].get('items'):
postJsonObject['object']['ignores']['items'] = []
itemsList = postJsonObject['object']['ignores']['items']
for ignoresItem in itemsList:
if ignoresItem.get('actor'):
if ignoresItem['actor'] == actor:
return
newIgnore = {
'type': 'Ignore',
'actor': actor
}
igIt = len(itemsList)
itemsList.append(newIgnore)
postJsonObject['object']['ignores']['totalItems'] = igIt
saveJson(postJsonObject, postFilename)
# remove cached post so that the muted version gets recreated
# without its content text and/or image
cachedPostFilename = \
getCachedPostFilename(baseDir, nickname, domain, postJsonObject)
if cachedPostFilename:
if os.path.isfile(cachedPostFilename):
os.remove(cachedPostFilename)
muteFile = open(postFilename + '.muted', 'w+')
if muteFile:
muteFile.write('\n')
muteFile.close()
print('MUTE: ' + postFilename + '.muted file added')
# if the post is in the recent posts cache then mark it as muted
if recentPostsCache.get('index'):
postId = \
removeIdEnding(postJsonObject['id']).replace('/', '#')
if postId in recentPostsCache['index']:
print('MUTE: ' + postId + ' is in recent posts cache')
if recentPostsCache['json'].get(postId):
postJsonObject['muted'] = True
recentPostsCache['json'][postId] = json.dumps(postJsonObject)
if recentPostsCache.get('html'):
if recentPostsCache['html'].get(postId):
del recentPostsCache['html'][postId]
print('MUTE: ' + postId +
' marked as muted in recent posts memory cache')
def unmutePost(baseDir: str, nickname: str, domain: str, port: int,
httpPrefix: str, postId: str, recentPostsCache: {},
debug: bool) -> None:
""" Unmutes the given post
"""
postFilename = locatePost(baseDir, nickname, domain, postId)
if not postFilename:
return
postJsonObject = loadJson(postFilename)
if not postJsonObject:
return
muteFilename = postFilename + '.muted'
if os.path.isfile(muteFilename):
os.remove(muteFilename)
print('UNMUTE: ' + muteFilename + ' file removed')
if postJsonObject.get('object'):
if isinstance(postJsonObject['object'], dict):
if postJsonObject['object'].get('ignores'):
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
totalItems = 0
if postJsonObject['object']['ignores'].get('totalItems'):
totalItems = \
postJsonObject['object']['ignores']['totalItems']
itemsList = postJsonObject['object']['ignores']['items']
for ignoresItem in itemsList:
if ignoresItem.get('actor'):
if ignoresItem['actor'] == actor:
if debug:
print('DEBUG: mute was removed for ' + actor)
itemsList.remove(ignoresItem)
break
if totalItems == 1:
if debug:
print('DEBUG: mute was removed from post')
del postJsonObject['object']['ignores']
else:
igItLen = len(postJsonObject['object']['ignores']['items'])
postJsonObject['object']['ignores']['totalItems'] = igItLen
saveJson(postJsonObject, postFilename)
# remove cached post so that the muted version gets recreated
# with its content text and/or image
cachedPostFilename = \
getCachedPostFilename(baseDir, nickname, domain, postJsonObject)
if cachedPostFilename:
if os.path.isfile(cachedPostFilename):
os.remove(cachedPostFilename)
# if the post is in the recent posts cache then mark it as unmuted
if recentPostsCache.get('index'):
postId = \
removeIdEnding(postJsonObject['id']).replace('/', '#')
if postId in recentPostsCache['index']:
print('UNMUTE: ' + postId + ' is in recent posts cache')
if recentPostsCache['json'].get(postId):
postJsonObject['muted'] = False
recentPostsCache['json'][postId] = json.dumps(postJsonObject)
if recentPostsCache.get('html'):
if recentPostsCache['html'].get(postId):
del recentPostsCache['html'][postId]
print('UNMUTE: ' + postId +
' marked as unmuted in recent posts cache')
def outboxMute(baseDir: str, httpPrefix: str,
nickname: str, domain: str, port: int,
messageJson: {}, debug: bool,
recentPostsCache: {}) -> None:
"""When a mute is received by the outbox from c2s
"""
if not messageJson.get('type'):
return
if not messageJson.get('actor'):
return
domainFull = getFullDomain(domain, port)
if not messageJson['actor'].endswith(domainFull + '/users/' + nickname):
return
if not messageJson['type'] == 'Ignore':
return
if not messageJson.get('object'):
if debug:
print('DEBUG: no object in mute')
return
if not isinstance(messageJson['object'], str):
if debug:
print('DEBUG: mute object is not string')
return
if debug:
print('DEBUG: c2s mute request arrived in outbox')
messageId = removeIdEnding(messageJson['object'])
if '/statuses/' not in messageId:
if debug:
print('DEBUG: c2s mute object is not a status')
return
if not hasUsersPath(messageId):
if debug:
print('DEBUG: c2s mute object has no nickname')
return
if ':' in domain:
domain = domain.split(':')[0]
postFilename = locatePost(baseDir, nickname, domain, messageId)
if not postFilename:
if debug:
print('DEBUG: c2s mute post not found in inbox or outbox')
print(messageId)
return
nicknameMuted = getNicknameFromActor(messageJson['object'])
if not nicknameMuted:
print('WARN: unable to find nickname in ' + messageJson['object'])
return
mutePost(baseDir, nickname, domain, port,
httpPrefix, messageJson['object'], recentPostsCache,
debug)
if debug:
print('DEBUG: post muted via c2s - ' + postFilename)
def outboxUndoMute(baseDir: str, httpPrefix: str,
nickname: str, domain: str, port: int,
messageJson: {}, debug: bool,
recentPostsCache: {}) -> None:
"""When an undo mute is received by the outbox from c2s
"""
if not messageJson.get('type'):
return
if not messageJson.get('actor'):
return
domainFull = getFullDomain(domain, port)
if not messageJson['actor'].endswith(domainFull + '/users/' + nickname):
return
if not messageJson['type'] == 'Undo':
return
if not messageJson.get('object'):
return
if not isinstance(messageJson['object'], dict):
return
if not messageJson['object'].get('type'):
return
if messageJson['object']['type'] != 'Ignore':
return
if not isinstance(messageJson['object']['object'], str):
if debug:
print('DEBUG: undo mute object is not a string')
return
if debug:
print('DEBUG: c2s undo mute request arrived in outbox')
messageId = removeIdEnding(messageJson['object']['object'])
if '/statuses/' not in messageId:
if debug:
print('DEBUG: c2s undo mute object is not a status')
return
if not hasUsersPath(messageId):
if debug:
print('DEBUG: c2s undo mute object has no nickname')
return
if ':' in domain:
domain = domain.split(':')[0]
postFilename = locatePost(baseDir, nickname, domain, messageId)
if not postFilename:
if debug:
print('DEBUG: c2s undo mute post not found in inbox or outbox')
print(messageId)
return
nicknameMuted = getNicknameFromActor(messageJson['object']['object'])
if not nicknameMuted:
print('WARN: unable to find nickname in ' +
messageJson['object']['object'])
return
unmutePost(baseDir, nickname, domain, port,
httpPrefix, messageJson['object']['object'],
recentPostsCache, debug)
if debug:
print('DEBUG: post undo mute via c2s - ' + postFilename)
def setBrochMode(baseDir: str, domainFull: str, enabled: bool) -> None:
"""Broch mode can be used to lock down the instance during
a period of time when it is temporarily under attack.

297
bookmarks.py
View File

@ -8,6 +8,8 @@ __status__ = "Production"
import os
from pprint import pprint
from webfinger import webfingerHandle
from auth import createBasicAuthHeader
from utils import hasUsersPath
from utils import getFullDomain
from utils import removeIdEnding
@ -19,6 +21,8 @@ from utils import locatePost
from utils import getCachedPostFilename
from utils import loadJson
from utils import saveJson
from posts import getPersonBox
from session import postJson
def undoBookmarksCollectionEntry(recentPostsCache: {},
@ -67,8 +71,8 @@ def undoBookmarksCollectionEntry(recentPostsCache: {},
return
if not postJsonObject.get('object'):
if debug:
pprint(postJsonObject)
print('DEBUG: post ' + objectUrl + ' has no object')
print('DEBUG: bookmarked post has no object ' +
str(postJsonObject))
return
if not isinstance(postJsonObject['object'], dict):
return
@ -154,11 +158,12 @@ def updateBookmarksCollection(recentPostsCache: {},
if not postJsonObject.get('object'):
if debug:
pprint(postJsonObject)
print('DEBUG: post ' + objectUrl + ' has no object')
print('DEBUG: no object in bookmarked post ' +
str(postJsonObject))
return
if not objectUrl.endswith('/bookmarks'):
objectUrl = objectUrl + '/bookmarks'
# does this post have bookmarks on it from differenent actors?
if not postJsonObject['object'].get('bookmarks'):
if debug:
print('DEBUG: Adding initial bookmarks to ' + objectUrl)
@ -341,6 +346,174 @@ def undoBookmark(recentPostsCache: {},
return newUndoBookmarkJson
def sendBookmarkViaServer(baseDir: str, session,
nickname: str, password: str,
domain: str, fromPort: int,
httpPrefix: str, bookmarkUrl: str,
cachedWebfingers: {}, personCache: {},
debug: bool, projectVersion: str) -> {}:
"""Creates a bookmark via c2s
"""
if not session:
print('WARN: No session for sendBookmarkViaServer')
return 6
domainFull = getFullDomain(domain, fromPort)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
newBookmarkJson = {
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Add",
"actor": actor,
"to": [actor],
"object": {
"type": "Document",
"url": bookmarkUrl,
"to": [actor]
},
"target": actor + "/tlbookmarks"
}
handle = httpPrefix + '://' + domainFull + '/@' + nickname
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: bookmark webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: bookmark webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
# get the actor inbox for the To handle
(inboxUrl, pubKeyId, pubKey, fromPersonId, sharedInbox,
avatarUrl, displayName) = getPersonBox(baseDir, session, wfRequest,
personCache,
projectVersion, httpPrefix,
nickname, domain,
postToBox, 52594)
if not inboxUrl:
if debug:
print('DEBUG: bookmark no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: bookmark no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(nickname, password)
headers = {
'host': domain,
'Content-type': 'application/json',
'Authorization': authHeader
}
postResult = postJson(session, newBookmarkJson, [], inboxUrl,
headers, 3, True)
if not postResult:
if debug:
print('WARN: POST bookmark failed for c2s to ' + inboxUrl)
return 5
if debug:
print('DEBUG: c2s POST bookmark success')
return newBookmarkJson
def sendUndoBookmarkViaServer(baseDir: str, session,
nickname: str, password: str,
domain: str, fromPort: int,
httpPrefix: str, bookmarkUrl: str,
cachedWebfingers: {}, personCache: {},
debug: bool, projectVersion: str) -> {}:
"""Removes a bookmark via c2s
"""
if not session:
print('WARN: No session for sendUndoBookmarkViaServer')
return 6
domainFull = getFullDomain(domain, fromPort)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
newBookmarkJson = {
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Remove",
"actor": actor,
"to": [actor],
"object": {
"type": "Document",
"url": bookmarkUrl,
"to": [actor]
},
"target": actor + "/tlbookmarks"
}
handle = httpPrefix + '://' + domainFull + '/@' + nickname
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: unbookmark webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: unbookmark webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
# get the actor inbox for the To handle
(inboxUrl, pubKeyId, pubKey, fromPersonId, sharedInbox,
avatarUrl, displayName) = getPersonBox(baseDir, session, wfRequest,
personCache,
projectVersion, httpPrefix,
nickname, domain,
postToBox, 52594)
if not inboxUrl:
if debug:
print('DEBUG: unbookmark no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: unbookmark no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(nickname, password)
headers = {
'host': domain,
'Content-type': 'application/json',
'Authorization': authHeader
}
postResult = postJson(session, newBookmarkJson, [], inboxUrl,
headers, 3, True)
if not postResult:
if debug:
print('WARN: POST unbookmark failed for c2s to ' + inboxUrl)
return 5
if debug:
print('DEBUG: c2s POST unbookmark success')
return newBookmarkJson
def outboxBookmark(recentPostsCache: {},
baseDir: str, httpPrefix: str,
nickname: str, domain: str, port: int,
@ -348,44 +521,63 @@ def outboxBookmark(recentPostsCache: {},
""" When a bookmark request is received by the outbox from c2s
"""
if not messageJson.get('type'):
if debug:
print('DEBUG: bookmark - no type')
return
if not messageJson['type'] == 'Bookmark':
if messageJson['type'] != 'Add':
return
if not messageJson.get('actor'):
if debug:
print('DEBUG: not a bookmark')
print('DEBUG: no actor in bookmark Add')
return
if not messageJson.get('object'):
if debug:
print('DEBUG: no object in bookmark')
print('DEBUG: no object in bookmark Add')
return
if not isinstance(messageJson['object'], str):
if not messageJson.get('target'):
if debug:
print('DEBUG: bookmark object is not string')
print('DEBUG: no target in bookmark Add')
return
if messageJson.get('to'):
if not isinstance(messageJson['to'], list):
if not isinstance(messageJson['object'], dict):
if debug:
print('DEBUG: bookmark Add object is not dict')
return
if len(messageJson['to']) != 1:
print('WARN: Bookmark should only be sent to one recipient')
if not messageJson['object'].get('type'):
if debug:
print('DEBUG: no object type in bookmark Add')
return
if messageJson['to'][0] != messageJson['actor']:
print('WARN: Bookmark should be addressed to the same actor')
if not isinstance(messageJson['target'], str):
if debug:
print('DEBUG: bookmark Add target is not string')
return
domainFull = getFullDomain(domain, port)
if not messageJson['target'].endswith('://' + domainFull +
'/users/' + nickname +
'/tlbookmarks'):
if debug:
print('DEBUG: bookmark Add target invalid ' +
messageJson['target'])
return
if messageJson['object']['type'] != 'Document':
if debug:
print('DEBUG: bookmark Add type is not Document')
return
if not messageJson['object'].get('url'):
if debug:
print('DEBUG: bookmark Add missing url')
return
if debug:
print('DEBUG: c2s bookmark request arrived in outbox')
print('DEBUG: c2s bookmark Add request arrived in outbox')
messageId = removeIdEnding(messageJson['object'])
messageUrl = removeIdEnding(messageJson['object']['url'])
if ':' in domain:
domain = domain.split(':')[0]
postFilename = locatePost(baseDir, nickname, domain, messageId)
postFilename = locatePost(baseDir, nickname, domain, messageUrl)
if not postFilename:
if debug:
print('DEBUG: c2s bookmark post not found in inbox or outbox')
print(messageId)
print('DEBUG: c2s like post not found in inbox or outbox')
print(messageUrl)
return True
updateBookmarksCollection(recentPostsCache,
baseDir, postFilename, messageId,
baseDir, postFilename, messageUrl,
messageJson['actor'], domain, debug)
if debug:
print('DEBUG: post bookmarked via c2s - ' + postFilename)
@ -399,53 +591,62 @@ def outboxUndoBookmark(recentPostsCache: {},
"""
if not messageJson.get('type'):
return
if not messageJson['type'] == 'Undo':
if messageJson['type'] != 'Remove':
return
if not messageJson.get('actor'):
if debug:
print('DEBUG: no actor in unbookmark Remove')
return
if not messageJson.get('object'):
if debug:
print('DEBUG: no object in unbookmark Remove')
return
if not messageJson.get('target'):
if debug:
print('DEBUG: no target in unbookmark Remove')
return
if not isinstance(messageJson['object'], dict):
if debug:
print('DEBUG: undo bookmark object is not dict')
print('DEBUG: unbookmark Remove object is not dict')
return
if not messageJson['object'].get('type'):
if debug:
print('DEBUG: undo bookmark - no type')
print('DEBUG: no object type in bookmark Remove')
return
if not messageJson['object']['type'] == 'Bookmark':
if not isinstance(messageJson['target'], str):
if debug:
print('DEBUG: not a undo bookmark')
print('DEBUG: unbookmark Remove target is not string')
return
if not messageJson['object'].get('object'):
domainFull = getFullDomain(domain, port)
if not messageJson['target'].endswith('://' + domainFull +
'/users/' + nickname +
'/tlbookmarks'):
if debug:
print('DEBUG: no object in undo bookmark')
print('DEBUG: unbookmark Remove target invalid ' +
messageJson['target'])
return
if not isinstance(messageJson['object']['object'], str):
if messageJson['object']['type'] != 'Document':
if debug:
print('DEBUG: undo bookmark object is not string')
print('DEBUG: unbookmark Remove type is not Document')
return
if messageJson.get('to'):
if not isinstance(messageJson['to'], list):
return
if len(messageJson['to']) != 1:
print('WARN: Bookmark should only be sent to one recipient')
return
if messageJson['to'][0] != messageJson['actor']:
print('WARN: Bookmark should be addressed to the same actor')
if not messageJson['object'].get('url'):
if debug:
print('DEBUG: unbookmark Remove missing url')
return
if debug:
print('DEBUG: c2s undo bookmark request arrived in outbox')
print('DEBUG: c2s unbookmark Remove request arrived in outbox')
messageId = removeIdEnding(messageJson['object']['object'])
messageUrl = removeIdEnding(messageJson['object']['url'])
if ':' in domain:
domain = domain.split(':')[0]
postFilename = locatePost(baseDir, nickname, domain, messageId)
postFilename = locatePost(baseDir, nickname, domain, messageUrl)
if not postFilename:
if debug:
print('DEBUG: c2s undo bookmark post not found in inbox or outbox')
print(messageId)
print('DEBUG: c2s unbookmark post not found in inbox or outbox')
print(messageUrl)
return True
undoBookmarksCollectionEntry(recentPostsCache,
baseDir, postFilename, messageId,
updateBookmarksCollection(recentPostsCache,
baseDir, postFilename, messageUrl,
messageJson['actor'], domain, debug)
if debug:
print('DEBUG: post undo bookmarked via c2s - ' + postFilename)
print('DEBUG: post unbookmarked via c2s - ' + postFilename)

4
content.py
View File

@ -643,6 +643,8 @@ def removeLongWords(content: str, maxWordLength: int,
if wordStr not in longWordsList:
longWordsList.append(wordStr)
for wordStr in longWordsList:
if wordStr.startswith('<p>'):
wordStr = wordStr.replace('<p>', '')
if wordStr.startswith('<'):
continue
if len(wordStr) == 76:
@ -678,6 +680,8 @@ def removeLongWords(content: str, maxWordLength: int,
continue
if '<' in wordStr:
replaceWord = wordStr.split('<', 1)[0]
# if len(replaceWord) > maxWordLength:
# replaceWord = replaceWord[:maxWordLength]
content = content.replace(wordStr, replaceWord)
wordStr = replaceWord
if '/' in wordStr:

64
daemon.py
View File

@ -10,7 +10,6 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer, HTTPServer
import sys
import json
import time
import locale
import urllib.parse
import datetime
from socket import error as SocketError
@ -74,8 +73,6 @@ from posts import pinPost
from posts import jsonPinPost
from posts import undoPinnedPost
from posts import isModerator
from posts import mutePost
from posts import unmutePost
from posts import createQuestionPost
from posts import createPublicPost
from posts import createBlogPost
@ -109,6 +106,8 @@ from threads import threadWithTrace
from threads import removeDormantThreads
from media import replaceYouTube
from media import attachMedia
from blocking import mutePost
from blocking import unmutePost
from blocking import setBrochMode
from blocking import addBlock
from blocking import removeBlock
@ -192,6 +191,7 @@ from shares import addShare
from shares import removeShare
from shares import expireShares
from categories import setHashtagCategory
from utils import loadTranslationsFromFile
from utils import getLocalNetworkAddresses
from utils import decodedHost
from utils import isPublicPost
@ -471,6 +471,8 @@ class PubServer(BaseHTTPRequestHandler):
postJsonObject['replies'] = {}
if postJsonObject.get('bookmarks'):
postJsonObject['bookmarks'] = {}
if postJsonObject.get('ignores'):
postJsonObject['ignores'] = {}
if not postJsonObject.get('object'):
return
if not isinstance(postJsonObject['object'], dict):
@ -483,6 +485,8 @@ class PubServer(BaseHTTPRequestHandler):
postJsonObject['object']['replies'] = {}
if postJsonObject['object'].get('bookmarks'):
postJsonObject['object']['bookmarks'] = {}
if postJsonObject['object'].get('ignores'):
postJsonObject['object']['ignores'] = {}
def _requestHTTP(self) -> bool:
"""Should a http response be given?
@ -1260,15 +1264,10 @@ class PubServer(BaseHTTPRequestHandler):
originalMessageJson = messageJson.copy()
# For follow activities add a 'to' field, which is a copy
# of the object field
addToFieldTypes = ('Follow', 'Like', 'Add', 'Remove', 'Ignore')
for addToType in addToFieldTypes:
messageJson, toFieldExists = \
addToField('Follow', messageJson, self.server.debug)
# For like activities add a 'to' field, which is a copy of
# the actor within the object field
messageJson, toFieldExists = \
addToField('Like', messageJson, self.server.debug)
addToField(addToType, messageJson, self.server.debug)
beginSaveTime = time.time()
# save the json for later queue processing
@ -7015,8 +7014,9 @@ class PubServer(BaseHTTPRequestHandler):
actor = \
httpPrefix + '://' + domainFull + path.split('?mute=')[0]
nickname = getNicknameFromActor(actor)
mutePost(baseDir, nickname, domain,
muteUrl, self.server.recentPostsCache)
mutePost(baseDir, nickname, domain, port,
httpPrefix, muteUrl,
self.server.recentPostsCache, debug)
self.server.GETbusy = False
if callingDomain.endswith('.onion') and onionDomain:
actor = \
@ -7059,8 +7059,9 @@ class PubServer(BaseHTTPRequestHandler):
actor = \
httpPrefix + '://' + domainFull + path.split('?unmute=')[0]
nickname = getNicknameFromActor(actor)
unmutePost(baseDir, nickname, domain,
muteUrl, self.server.recentPostsCache)
unmutePost(baseDir, nickname, domain, port,
httpPrefix, muteUrl,
self.server.recentPostsCache, debug)
self.server.GETbusy = False
if callingDomain.endswith('.onion') and onionDomain:
actor = \
@ -14443,32 +14444,11 @@ def runDaemon(brochMode: bool,
httpd.translate = {}
httpd.systemLanguage = 'en'
if not unitTest:
if not os.path.isdir(baseDir + '/translations'):
print('ERROR: translations directory not found')
return
if not language:
systemLanguage = locale.getdefaultlocale()[0]
else:
systemLanguage = language
if not systemLanguage:
systemLanguage = 'en'
if '_' in systemLanguage:
systemLanguage = systemLanguage.split('_')[0]
while '/' in systemLanguage:
systemLanguage = systemLanguage.split('/')[1]
if '.' in systemLanguage:
systemLanguage = systemLanguage.split('.')[0]
translationsFile = baseDir + '/translations/' + \
systemLanguage + '.json'
if not os.path.isfile(translationsFile):
systemLanguage = 'en'
translationsFile = baseDir + '/translations/' + \
systemLanguage + '.json'
print('System language: ' + systemLanguage)
httpd.systemLanguage = systemLanguage
httpd.translate = loadJson(translationsFile)
httpd.translate, httpd.systemLanguage = \
loadTranslationsFromFile(baseDir, language)
print('System language: ' + httpd.systemLanguage)
if not httpd.translate:
print('ERROR: no translations loaded from ' + translationsFile)
print('ERROR: no translations were loaded')
sys.exit()
# For moderated newswire feeds this is the amount of time allowed
@ -14546,8 +14526,8 @@ def runDaemon(brochMode: bool,
# max POST size of 30M
httpd.maxPostLength = 1024 * 1024 * 30
httpd.maxMediaSize = httpd.maxPostLength
# Maximum text length is 32K - enough for a blog post
httpd.maxMessageLength = 32000
# Maximum text length is 64K - enough for a blog post
httpd.maxMessageLength = 64000
# Maximum overall number of posts per box
httpd.maxPostsInBox = 32000
httpd.domain = domain

15
delete.py
View File

@ -58,11 +58,11 @@ def sendDeleteViaServer(baseDir: str, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: delete webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: delete webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -76,11 +76,12 @@ def sendDeleteViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: delete no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: delete no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -91,10 +92,10 @@ def sendDeleteViaServer(baseDir: str, session,
'Authorization': authHeader
}
postResult = \
postJson(session, newDeleteJson, [], inboxUrl, headers, 30, True)
postJson(session, newDeleteJson, [], inboxUrl, headers, 3, True)
if not postResult:
if debug:
print('DEBUG: POST announce failed for c2s to ' + inboxUrl)
print('DEBUG: POST delete failed for c2s to ' + inboxUrl)
return 5
if debug:

1920
desktop_client.py 100644
View File

File diff suppressed because it is too large Load Diff

201
epicyon.py
View File

@ -22,6 +22,11 @@ from person import deactivateAccount
from skills import setSkillLevel
from roles import setRole
from webfinger import webfingerHandle
from bookmarks import sendBookmarkViaServer
from bookmarks import sendUndoBookmarkViaServer
from posts import sendMuteViaServer
from posts import sendUndoMuteViaServer
from posts import c2sBoxJson
from posts import downloadFollowCollection
from posts import getPublicPostDomains
from posts import getPublicPostDomainsBlocked
@ -48,6 +53,7 @@ from follow import sendUnfollowRequestViaServer
from tests import testPostMessageBetweenServers
from tests import testFollowBetweenServers
from tests import testClientToServer
from tests import testUpdateActor
from tests import runAllTests
from auth import storeBasicCredentials
from auth import createPassword
@ -78,7 +84,7 @@ from theme import setTheme
from announce import sendAnnounceViaServer
from socnet import instancesGraph
from migrate import migrateAccounts
from notifications_client import runNotificationsClient
from desktop_client import runDesktopClient
def str2bool(v) -> bool:
@ -304,7 +310,7 @@ parser.add_argument("--notifyShowNewPosts",
dest='notifyShowNewPosts',
type=str2bool, nargs='?',
const=True, default=False,
help="Notification client shows/speaks new posts " +
help="Desktop client shows/speaks new posts " +
"as they arrive")
parser.add_argument("--noapproval", type=str2bool, nargs='?',
const=True, default=False,
@ -404,10 +410,23 @@ parser.add_argument("--allowdeletion", type=str2bool, nargs='?',
parser.add_argument('--repeat', '--announce', dest='announce', type=str,
default=None,
help='Announce/repeat a url')
parser.add_argument('--box', type=str,
default=None,
help='Returns the json for a given timeline, ' +
'with authentication')
parser.add_argument('--page', '--pageNumber', dest='pageNumber', type=int,
default=1,
help='Page number when using the --box option')
parser.add_argument('--favorite', '--like', dest='like', type=str,
default=None, help='Like a url')
parser.add_argument('--undolike', '--unlike', dest='undolike', type=str,
default=None, help='Undo a like of a url')
parser.add_argument('--bookmark', '--bm', dest='bookmark', type=str,
default=None,
help='Bookmark the url of a post')
parser.add_argument('--unbookmark', '--unbm', dest='unbookmark', type=str,
default=None,
help='Undo a bookmark given the url of a post')
parser.add_argument('--sendto', dest='sendto', type=str,
default=None, help='Address to send a post to')
parser.add_argument('--attach', dest='attach', type=str,
@ -461,6 +480,10 @@ parser.add_argument('--block', dest='block', type=str, default=None,
help='Block a particular address')
parser.add_argument('--unblock', dest='unblock', type=str, default=None,
help='Remove a block on a particular address')
parser.add_argument('--mute', dest='mute', type=str, default=None,
help='Mute a particular post URL')
parser.add_argument('--unmute', dest='unmute', type=str, default=None,
help='Unmute a particular post URL')
parser.add_argument('--delegate', dest='delegate', type=str, default=None,
help='Address of an account to delegate a role to')
parser.add_argument('--undodelegate', '--undelegate', dest='undelegate',
@ -531,6 +554,7 @@ if args.testsnetwork:
testPostMessageBetweenServers()
testFollowBetweenServers()
testClientToServer()
testUpdateActor()
print('All tests succeeded')
sys.exit()
@ -1110,6 +1134,46 @@ if args.announce:
time.sleep(1)
sys.exit()
if args.box:
if not domain:
print('Specify a domain with the --domain option')
sys.exit()
if not args.nickname:
print('Specify a nickname with the --nickname option')
sys.exit()
if not args.password:
args.password = getpass.getpass('Password: ')
if not args.password:
print('Specify a password with the --password option')
sys.exit()
args.password = args.password.replace('\n', '')
proxyType = None
if args.tor or domain.endswith('.onion'):
proxyType = 'tor'
if domain.endswith('.onion'):
args.port = 80
elif args.i2p or domain.endswith('.i2p'):
proxyType = 'i2p'
if domain.endswith('.i2p'):
args.port = 80
elif args.gnunet:
proxyType = 'gnunet'
session = createSession(proxyType)
boxJson = c2sBoxJson(baseDir, session,
args.nickname, args.password,
domain, port, httpPrefix,
args.box, args.pageNumber,
args.debug)
if boxJson:
pprint(boxJson)
else:
print('Box not found: ' + args.box)
sys.exit()
if args.itemName:
if not args.password:
args.password = getpass.getpass('Password: ')
@ -1254,6 +1318,62 @@ if args.undolike:
time.sleep(1)
sys.exit()
if args.bookmark:
if not args.nickname:
print('Specify a nickname with the --nickname option')
sys.exit()
if not args.password:
args.password = getpass.getpass('Password: ')
if not args.password:
print('Specify a password with the --password option')
sys.exit()
args.password = args.password.replace('\n', '')
session = createSession(proxyType)
personCache = {}
cachedWebfingers = {}
print('Sending bookmark of ' + args.bookmark)
sendBookmarkViaServer(baseDir, session,
args.nickname, args.password,
domain, port,
httpPrefix, args.bookmark,
cachedWebfingers, personCache,
True, __version__)
for i in range(10):
# TODO detect send success/fail
time.sleep(1)
sys.exit()
if args.unbookmark:
if not args.nickname:
print('Specify a nickname with the --nickname option')
sys.exit()
if not args.password:
args.password = getpass.getpass('Password: ')
if not args.password:
print('Specify a password with the --password option')
sys.exit()
args.password = args.password.replace('\n', '')
session = createSession(proxyType)
personCache = {}
cachedWebfingers = {}
print('Sending undo bookmark of ' + args.unbookmark)
sendUndoBookmarkViaServer(baseDir, session,
args.nickname, args.password,
domain, port,
httpPrefix, args.unbookmark,
cachedWebfingers, personCache,
True, __version__)
for i in range(10):
# TODO detect send success/fail
time.sleep(1)
sys.exit()
if args.delete:
if not args.nickname:
print('Specify a nickname with the --nickname option')
@ -1411,7 +1531,7 @@ if args.migrations:
sys.exit()
if args.actor:
getActorJson(args.actor, args.http, args.gnunet, False)
getActorJson(args.actor, args.http, args.gnunet, debug)
sys.exit()
if args.followers:
@ -1868,7 +1988,7 @@ if args.desktop:
# only store inbox posts if we are not running as a daemon
storeInboxPosts = not args.noKeyPress
runNotificationsClient(baseDir, proxyType, httpPrefix,
runDesktopClient(baseDir, proxyType, httpPrefix,
nickname, domain, port, args.password,
args.screenreader, args.language,
args.notificationSounds,
@ -1876,6 +1996,7 @@ if args.desktop:
args.noKeyPress,
storeInboxPosts,
args.notifyShowNewPosts,
args.language,
args.debug)
sys.exit()
@ -1921,6 +2042,60 @@ if args.block:
time.sleep(1)
sys.exit()
if args.mute:
if not nickname:
print('Specify a nickname with the --nickname option')
sys.exit()
if not args.password:
args.password = getpass.getpass('Password: ')
if not args.password:
print('Specify a password with the --password option')
sys.exit()
args.password = args.password.replace('\n', '')
session = createSession(proxyType)
personCache = {}
cachedWebfingers = {}
print('Sending mute of ' + args.mute)
sendMuteViaServer(baseDir, session, nickname, args.password,
domain, port,
httpPrefix, args.mute,
cachedWebfingers, personCache,
True, __version__)
for i in range(10):
# TODO detect send success/fail
time.sleep(1)
sys.exit()
if args.unmute:
if not nickname:
print('Specify a nickname with the --nickname option')
sys.exit()
if not args.password:
args.password = getpass.getpass('Password: ')
if not args.password:
print('Specify a password with the --password option')
sys.exit()
args.password = args.password.replace('\n', '')
session = createSession(proxyType)
personCache = {}
cachedWebfingers = {}
print('Sending undo mute of ' + args.unmute)
sendUndoMuteViaServer(baseDir, session, nickname, args.password,
domain, port,
httpPrefix, args.unmute,
cachedWebfingers, personCache,
True, __version__)
for i in range(10):
# TODO detect send success/fail
time.sleep(1)
sys.exit()
if args.delegate:
if not nickname:
print('Specify a nickname with the --nickname option')
@ -2121,7 +2296,7 @@ if args.testdata:
testFollowersOnly = False
testSaveToFile = True
testClientToServer = False
testC2S = False
testCommentsEnabled = True
testAttachImageFilename = None
testMediaType = None
@ -2131,7 +2306,7 @@ if args.testdata:
"like this is totally just a #test man",
testFollowersOnly,
testSaveToFile,
testClientToServer,
testC2S,
testCommentsEnabled,
testAttachImageFilename,
testMediaType, testImageDescription)
@ -2139,7 +2314,7 @@ if args.testdata:
"Zoiks!!!",
testFollowersOnly,
testSaveToFile,
testClientToServer,
testC2S,
testCommentsEnabled,
testAttachImageFilename,
testMediaType, testImageDescription)
@ -2147,7 +2322,7 @@ if args.testdata:
"Hey scoob we need like a hundred more #milkshakes",
testFollowersOnly,
testSaveToFile,
testClientToServer,
testC2S,
testCommentsEnabled,
testAttachImageFilename,
testMediaType, testImageDescription)
@ -2155,7 +2330,7 @@ if args.testdata:
"Getting kinda spooky around here",
testFollowersOnly,
testSaveToFile,
testClientToServer,
testC2S,
testCommentsEnabled,
testAttachImageFilename,
testMediaType, testImageDescription,
@ -2165,7 +2340,7 @@ if args.testdata:
"if it wasn't for those pesky hackers",
testFollowersOnly,
testSaveToFile,
testClientToServer,
testC2S,
testCommentsEnabled,
'img/logo.png', 'image/png',
'Description of image')
@ -2173,7 +2348,7 @@ if args.testdata:
"man these centralized sites are like the worst!",
testFollowersOnly,
testSaveToFile,
testClientToServer,
testC2S,
testCommentsEnabled,
testAttachImageFilename,
testMediaType, testImageDescription)
@ -2181,7 +2356,7 @@ if args.testdata:
"another mystery solved #test",
testFollowersOnly,
testSaveToFile,
testClientToServer,
testC2S,
testCommentsEnabled,
testAttachImageFilename,
testMediaType, testImageDescription)
@ -2189,7 +2364,7 @@ if args.testdata:
"let's go bowling",
testFollowersOnly,
testSaveToFile,
testClientToServer,
testC2S,
testCommentsEnabled,
testAttachImageFilename,
testMediaType, testImageDescription)

32
follow.py
View File

@ -992,11 +992,11 @@ def sendFollowRequestViaServer(baseDir: str, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: follow request webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: follow request Webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -1010,11 +1010,12 @@ def sendFollowRequestViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: follow request no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: follow request no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -1025,14 +1026,14 @@ def sendFollowRequestViaServer(baseDir: str, session,
'Authorization': authHeader
}
postResult = \
postJson(session, newFollowJson, [], inboxUrl, headers, 30, True)
postJson(session, newFollowJson, [], inboxUrl, headers, 3, True)
if not postResult:
if debug:
print('DEBUG: POST announce failed for c2s to ' + inboxUrl)
print('DEBUG: POST follow request failed for c2s to ' + inboxUrl)
return 5
if debug:
print('DEBUG: c2s POST follow success')
print('DEBUG: c2s POST follow request success')
return newFollowJson
@ -1081,11 +1082,11 @@ def sendUnfollowRequestViaServer(baseDir: str, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: unfollow webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: unfollow webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -1102,11 +1103,12 @@ def sendUnfollowRequestViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: unfollow no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: unfollow no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -1117,10 +1119,10 @@ def sendUnfollowRequestViaServer(baseDir: str, session,
'Authorization': authHeader
}
postResult = \
postJson(session, unfollowJson, [], inboxUrl, headers, 30, True)
postJson(session, unfollowJson, [], inboxUrl, headers, 3, True)
if not postResult:
if debug:
print('DEBUG: POST announce failed for c2s to ' + inboxUrl)
print('DEBUG: POST unfollow failed for c2s to ' + inboxUrl)
return 5
if debug:

156
inbox.py
View File

@ -1082,60 +1082,73 @@ def _receiveBookmark(recentPostsCache: {},
debug: bool) -> bool:
"""Receives a bookmark activity within the POST section of HTTPServer
"""
if messageJson['type'] != 'Bookmark':
if not messageJson.get('type'):
return False
if messageJson['type'] != 'Add':
return False
if not messageJson.get('actor'):
if debug:
print('DEBUG: ' + messageJson['type'] + ' has no actor')
print('DEBUG: no actor in inbox bookmark Add')
return False
if not messageJson.get('object'):
if debug:
print('DEBUG: ' + messageJson['type'] + ' has no object')
print('DEBUG: no object in inbox bookmark Add')
return False
if not isinstance(messageJson['object'], str):
if not messageJson.get('target'):
if debug:
print('DEBUG: ' + messageJson['type'] + ' object is not a string')
print('DEBUG: no target in inbox bookmark Add')
return False
if not messageJson.get('to'):
if not isinstance(messageJson['object'], dict):
if debug:
print('DEBUG: ' + messageJson['type'] + ' has no "to" list')
print('DEBUG: inbox bookmark Add object is not string')
return False
if '/users/' not in messageJson['actor']:
if not messageJson['object'].get('type'):
if debug:
print('DEBUG: "users" missing from actor in ' +
messageJson['type'])
print('DEBUG: no object type in inbox bookmark Add')
return False
if '/statuses/' not in messageJson['object']:
if not isinstance(messageJson['target'], str):
if debug:
print('DEBUG: "statuses" missing from object in ' +
messageJson['type'])
return False
if domain not in handle.split('@')[1]:
if debug:
print('DEBUG: unrecognized domain ' + handle)
print('DEBUG: inbox bookmark Add target is not string')
return False
domainFull = getFullDomain(domain, port)
nickname = handle.split('@')[0]
if not messageJson['actor'].endswith(domainFull + '/users/' + nickname):
if debug:
print('DEBUG: ' +
'bookmark actor should be the same as the handle sent to ' +
handle + ' != ' + messageJson['actor'])
print('DEBUG: inbox bookmark Add unexpected actor')
return False
if not os.path.isdir(baseDir + '/accounts/' + handle):
print('DEBUG: unknown recipient of bookmark - ' + handle)
# if this post in the outbox of the person?
postFilename = locatePost(baseDir, nickname, domain, messageJson['object'])
if not messageJson['target'].endswith(messageJson['actor'] +
'/tlbookmarks'):
if debug:
print('DEBUG: inbox bookmark Add target invalid ' +
messageJson['target'])
return False
if messageJson['object']['type'] != 'Document':
if debug:
print('DEBUG: inbox bookmark Add type is not Document')
return False
if not messageJson['object'].get('url'):
if debug:
print('DEBUG: inbox bookmark Add missing url')
return False
if '/statuses/' not in messageJson['object']['url']:
if debug:
print('DEBUG: inbox bookmark Add missing statuses un url')
return False
if debug:
print('DEBUG: c2s inbox bookmark Add request arrived in outbox')
messageUrl = removeIdEnding(messageJson['object']['url'])
if ':' in domain:
domain = domain.split(':')[0]
postFilename = locatePost(baseDir, nickname, domain, messageUrl)
if not postFilename:
if debug:
print('DEBUG: post not found in inbox or outbox')
print(messageJson['object'])
print('DEBUG: c2s inbox like post not found in inbox or outbox')
print(messageUrl)
return True
if debug:
print('DEBUG: bookmarked post was found')
updateBookmarksCollection(recentPostsCache, baseDir, postFilename,
messageJson['object'],
messageJson['object']['url'],
messageJson['actor'], domain, debug)
return True
@ -1148,63 +1161,74 @@ def _receiveUndoBookmark(recentPostsCache: {},
debug: bool) -> bool:
"""Receives an undo bookmark activity within the POST section of HTTPServer
"""
if messageJson['type'] != 'Undo':
if not messageJson.get('type'):
return False
if messageJson['type'] != 'Remove':
return False
if not messageJson.get('actor'):
if debug:
print('DEBUG: no actor in inbox undo bookmark Remove')
return False
if not messageJson.get('object'):
if debug:
print('DEBUG: no object in inbox undo bookmark Remove')
return False
if not messageJson.get('target'):
if debug:
print('DEBUG: no target in inbox undo bookmark Remove')
return False
if not isinstance(messageJson['object'], dict):
if debug:
print('DEBUG: inbox Remove bookmark object is not dict')
return False
if not messageJson['object'].get('type'):
return False
if messageJson['object']['type'] != 'Bookmark':
return False
if not messageJson['object'].get('object'):
if debug:
print('DEBUG: ' + messageJson['type'] + ' like has no object')
print('DEBUG: no object type in inbox bookmark Remove')
return False
if not isinstance(messageJson['object']['object'], str):
if not isinstance(messageJson['target'], str):
if debug:
print('DEBUG: ' + messageJson['type'] +
' like object is not a string')
return False
if '/users/' not in messageJson['actor']:
if debug:
print('DEBUG: "users" missing from actor in ' +
messageJson['type'] + ' like')
return False
if '/statuses/' not in messageJson['object']['object']:
if debug:
print('DEBUG: "statuses" missing from like object in ' +
messageJson['type'])
print('DEBUG: inbox Remove bookmark target is not string')
return False
domainFull = getFullDomain(domain, port)
nickname = handle.split('@')[0]
if domain not in handle.split('@')[1]:
if debug:
print('DEBUG: unrecognized bookmark domain ' + handle)
return False
if not messageJson['actor'].endswith(domainFull + '/users/' + nickname):
if debug:
print('DEBUG: ' +
'bookmark actor should be the same as the handle sent to ' +
handle + ' != ' + messageJson['actor'])
print('DEBUG: inbox undo bookmark Remove unexpected actor')
return False
if not os.path.isdir(baseDir + '/accounts/' + handle):
print('DEBUG: unknown recipient of bookmark undo - ' + handle)
# if this post in the outbox of the person?
postFilename = locatePost(baseDir, nickname, domain,
messageJson['object']['object'])
if not messageJson['target'].endswith(messageJson['actor'] +
'/tlbookmarks'):
if debug:
print('DEBUG: inbox undo bookmark Remove target invalid ' +
messageJson['target'])
return False
if messageJson['object']['type'] != 'Document':
if debug:
print('DEBUG: inbox undo bookmark Remove type is not Document')
return False
if not messageJson['object'].get('url'):
if debug:
print('DEBUG: inbox undo bookmark Remove missing url')
return False
if '/statuses/' not in messageJson['object']['url']:
if debug:
print('DEBUG: inbox undo bookmark Remove missing statuses un url')
return False
if debug:
print('DEBUG: c2s inbox Remove bookmark ' +
'request arrived in outbox')
messageUrl = removeIdEnding(messageJson['object']['url'])
if ':' in domain:
domain = domain.split(':')[0]
postFilename = locatePost(baseDir, nickname, domain, messageUrl)
if not postFilename:
if debug:
print('DEBUG: unbookmarked post not found in inbox or outbox')
print(messageJson['object']['object'])
print('DEBUG: c2s inbox like post not found in inbox or outbox')
print(messageUrl)
return True
if debug:
print('DEBUG: bookmarked post found. Now undoing.')
undoBookmarksCollectionEntry(recentPostsCache, baseDir, postFilename,
messageJson['object'],
messageJson['object']['url'],
messageJson['actor'], domain, debug)
return True
@ -1588,10 +1612,12 @@ def populateReplies(baseDir: str, httpPrefix: str, domain: str,
return False
if messageId not in open(postRepliesFilename).read():
repliesFile = open(postRepliesFilename, 'a+')
if repliesFile:
repliesFile.write(messageId + '\n')
repliesFile.close()
else:
repliesFile = open(postRepliesFilename, 'w+')
if repliesFile:
repliesFile.write(messageId + '\n')
repliesFile.close()
return True

28
like.py
View File

@ -170,11 +170,11 @@ def sendLikeViaServer(baseDir: str, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: like webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: like webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -189,11 +189,11 @@ def sendLikeViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: like no ' + postToBox + ' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: like no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -204,10 +204,10 @@ def sendLikeViaServer(baseDir: str, session,
'Authorization': authHeader
}
postResult = postJson(session, newLikeJson, [], inboxUrl,
headers, 30, True)
headers, 3, True)
if not postResult:
if debug:
print('WARN: POST announce failed for c2s to ' + inboxUrl)
print('WARN: POST like failed for c2s to ' + inboxUrl)
return 5
if debug:
@ -251,11 +251,11 @@ def sendUndoLikeViaServer(baseDir: str, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: unlike webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
if debug:
print('WARN: Webfinger for ' + handle +
print('WARN: unlike webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
@ -271,11 +271,11 @@ def sendUndoLikeViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: unlike no ' + postToBox + ' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: unlike no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -286,14 +286,14 @@ def sendUndoLikeViaServer(baseDir: str, session,
'Authorization': authHeader
}
postResult = postJson(session, newUndoLikeJson, [], inboxUrl,
headers, 30, True)
headers, 3, True)
if not postResult:
if debug:
print('WARN: POST announce failed for c2s to ' + inboxUrl)
print('WARN: POST unlike failed for c2s to ' + inboxUrl)
return 5
if debug:
print('DEBUG: c2s POST undo like success')
print('DEBUG: c2s POST unlike success')
return newUndoLikeJson

1388
notifications_client.py
View File

File diff suppressed because it is too large Load Diff

147
outbox.py
View File

@ -21,9 +21,13 @@ from utils import removeIdEnding
from utils import getDomainFromActor
from utils import dangerousMarkup
from utils import isFeaturedWriter
from utils import loadJson
from utils import saveJson
from blocking import isBlockedDomain
from blocking import outboxBlock
from blocking import outboxUndoBlock
from blocking import outboxMute
from blocking import outboxUndoMute
from media import replaceYouTube
from media import getMediaPath
from media import createMediaDirs
@ -42,6 +46,123 @@ from shares import outboxShareUpload
from shares import outboxUndoShareUpload
def _outboxPersonReceiveUpdate(recentPostsCache: {},
baseDir: str, httpPrefix: str,
nickname: str, domain: str, port: int,
messageJson: {}, debug: bool) -> None:
""" Receive an actor update from c2s
For example, setting the PGP key from the desktop client
"""
# these attachments are updatable via c2s
updatableAttachments = ('PGP', 'OpenPGP', 'Email')
if not messageJson.get('type'):
return
print("messageJson['type'] " + messageJson['type'])
if messageJson['type'] != 'Update':
return
if not messageJson.get('object'):
return
if not isinstance(messageJson['object'], dict):
if debug:
print('DEBUG: c2s actor update object is not dict')
return
if not messageJson['object'].get('type'):
if debug:
print('DEBUG: c2s actor update - no type')
return
if messageJson['object']['type'] != 'Person':
if debug:
print('DEBUG: not a c2s actor update')
return
if not messageJson.get('to'):
if debug:
print('DEBUG: c2s actor update has no "to" field')
return
if not messageJson.get('actor'):
if debug:
print('DEBUG: c2s actor update has no actor field')
return
if not messageJson.get('id'):
if debug:
print('DEBUG: c2s actor update has no id field')
return
actor = \
httpPrefix + '://' + getFullDomain(domain, port) + '/users/' + nickname
if len(messageJson['to']) != 1:
if debug:
print('DEBUG: c2s actor update - to does not contain one actor ' +
messageJson['to'])
return
if messageJson['to'][0] != actor:
if debug:
print('DEBUG: c2s actor update - to does not contain actor ' +
messageJson['to'] + ' ' + actor)
return
if not messageJson['id'].startswith(actor + '#updates/'):
if debug:
print('DEBUG: c2s actor update - unexpected id ' +
messageJson['id'])
return
updatedActorJson = messageJson['object']
# load actor from file
actorFilename = baseDir + '/accounts/' + nickname + '@' + domain + '.json'
if not os.path.isfile(actorFilename):
print('actorFilename not found: ' + actorFilename)
return
actorJson = loadJson(actorFilename)
if not actorJson:
return
actorChanged = False
# update fields within actor
if 'attachment' in updatedActorJson:
for newPropertyValue in updatedActorJson['attachment']:
if not newPropertyValue.get('name'):
continue
if newPropertyValue['name'] not in updatableAttachments:
continue
if not newPropertyValue.get('type'):
continue
if not newPropertyValue.get('value'):
continue
if newPropertyValue['type'] != 'PropertyValue':
continue
if 'attachment' in actorJson:
found = False
for attachIdx in range(len(actorJson['attachment'])):
if actorJson['attachment'][attachIdx]['type'] != \
'PropertyValue':
continue
if actorJson['attachment'][attachIdx]['name'] != \
newPropertyValue['name']:
continue
else:
if actorJson['attachment'][attachIdx]['value'] != \
newPropertyValue['value']:
actorJson['attachment'][attachIdx]['value'] = \
newPropertyValue['value']
actorChanged = True
found = True
break
if not found:
actorJson['attachment'].append({
"name": newPropertyValue['name'],
"type": "PropertyValue",
"value": newPropertyValue['value']
})
actorChanged = True
# save actor to file
if actorChanged:
saveJson(actorJson, actorFilename)
if debug:
print('actor saved: ' + actorFilename)
if debug:
print('New attachment: ' + str(actorJson['attachment']))
messageJson['object'] = actorJson
if debug:
print('DEBUG: actor update via c2s - ' + nickname + '@' + domain)
def postMessageToOutbox(session, translate: {},
messageJson: {}, postToNickname: str,
server, baseDir: str, httpPrefix: str,
@ -190,7 +311,8 @@ def postMessageToOutbox(session, translate: {},
permittedOutboxTypes = ('Create', 'Announce', 'Like', 'Follow', 'Undo',
'Update', 'Add', 'Remove', 'Block', 'Delete',
'Delegate', 'Skill', 'Bookmark', 'Event')
'Delegate', 'Skill', 'Add', 'Remove', 'Event',
'Ignore')
if messageJson['type'] not in permittedOutboxTypes:
if debug:
print('DEBUG: POST to outbox - ' + messageJson['type'] +
@ -396,6 +518,22 @@ def postMessageToOutbox(session, translate: {},
postToNickname, domain,
port, messageJson, debug)
if debug:
print('DEBUG: handle mute requests')
outboxMute(baseDir, httpPrefix,
postToNickname, domain,
port,
messageJson, debug,
recentPostsCache)
if debug:
print('DEBUG: handle undo mute requests')
outboxUndoMute(baseDir, httpPrefix,
postToNickname, domain,
port,
messageJson, debug,
recentPostsCache)
if debug:
print('DEBUG: handle share uploads')
outboxShareUpload(baseDir, httpPrefix,
@ -408,6 +546,13 @@ def postMessageToOutbox(session, translate: {},
postToNickname, domain,
port, messageJson, debug)
if debug:
print('DEBUG: handle actor updates from c2s')
_outboxPersonReceiveUpdate(recentPostsCache,
baseDir, httpPrefix,
postToNickname, domain, port,
messageJson, debug)
if debug:
print('DEBUG: sending c2s post to named addresses')
if messageJson.get('to'):

27
person.py
View File

@ -1106,6 +1106,8 @@ def getActorJson(handle: str, http: bool, gnunet: bool,
debug: bool, quiet=False) -> {}:
"""Returns the actor json
"""
if debug:
print('getActorJson for ' + handle)
originalActor = handle
if '/@' in handle or \
'/users/' in handle or \
@ -1117,8 +1119,8 @@ def getActorJson(handle: str, http: bool, gnunet: bool,
handle = handle.replace(prefix, '')
handle = handle.replace('/@', '/users/')
if not hasUsersPath(handle):
if not quiet:
print('Expected actor format: ' +
if not quiet or debug:
print('getActorJson: Expected actor format: ' +
'https://domain/@nick or https://domain/users/nick')
return None
if '/users/' in handle:
@ -1145,13 +1147,13 @@ def getActorJson(handle: str, http: bool, gnunet: bool,
# format: @nick@domain
if '@' not in handle:
if not quiet:
print('Syntax: --actor nickname@domain')
print('getActorJson Syntax: --actor nickname@domain')
return None
if handle.startswith('@'):
handle = handle[1:]
if '@' not in handle:
if not quiet:
print('Syntax: --actor nickname@domain')
print('getActorJsonSyntax: --actor nickname@domain')
return None
nickname = handle.split('@')[0]
domain = handle.split('@')[1]
@ -1168,7 +1170,10 @@ def getActorJson(handle: str, http: bool, gnunet: bool,
httpPrefix = 'gnunet'
proxyType = 'gnunet'
else:
if '127.0.' not in domain and '192.168.' not in domain:
httpPrefix = 'https'
else:
httpPrefix = 'http'
session = createSession(proxyType)
if nickname == 'inbox':
nickname = domain
@ -1179,12 +1184,12 @@ def getActorJson(handle: str, http: bool, gnunet: bool,
None, __version__, debug)
if not wfRequest:
if not quiet:
print('Unable to webfinger ' + handle)
print('getActorJson Unable to webfinger ' + handle)
return None
if not isinstance(wfRequest, dict):
if not quiet:
print('Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('getActorJson Webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return None
if not quiet:
@ -1192,11 +1197,13 @@ def getActorJson(handle: str, http: bool, gnunet: bool,
personUrl = None
if wfRequest.get('errors'):
if not quiet:
print('wfRequest error: ' + str(wfRequest['errors']))
if not quiet or debug:
print('getActorJson wfRequest error: ' + str(wfRequest['errors']))
if hasUsersPath(handle):
personUrl = originalActor
else:
if debug:
print('No users path in ' + handle)
return None
profileStr = 'https://www.w3.org/ns/activitystreams'
@ -1230,6 +1237,7 @@ def getActorJson(handle: str, http: bool, gnunet: bool,
if personJson:
if not quiet:
pprint(personJson)
return personJson
else:
profileStr = 'https://www.w3.org/ns/activitystreams'
asHeader = {
@ -1240,6 +1248,7 @@ def getActorJson(handle: str, http: bool, gnunet: bool,
debug, __version__, httpPrefix, None)
if not quiet:
if personJson:
print('getActorJson returned actor')
pprint(personJson)
else:
print('Failed to get ' + personUrl)

205
pgp.py
View File

@ -12,6 +12,12 @@ from pathlib import Path
from person import getActorJson
from utils import containsPGPPublicKey
from utils import isPGPEncrypted
from utils import getFullDomain
from utils import getStatusNumber
from webfinger import webfingerHandle
from posts import getPersonBox
from auth import createBasicAuthHeader
from session import postJson
def getEmailAddress(actorJson: {}) -> str:
@ -330,7 +336,7 @@ def _getPGPPublicKeyFromActor(handle: str, actorJson=None) -> str:
public key specified
"""
if not actorJson:
actorJson = getActorJson(handle, False, False, True)
actorJson = getActorJson(handle, False, False, False, True)
if not actorJson:
return None
if not actorJson.get('attachment'):
@ -395,3 +401,200 @@ def pgpDecrypt(content: str, fromHandle: str) -> str:
return content
decryptResult = decryptResult.decode('utf-8').strip()
return decryptResult
def _pgpLocalPublicKeyId() -> str:
"""Gets the local pgp public key ID
"""
cmdStr = \
"gpgconf --list-options gpg | " + \
"awk -F: '$1 == \"default-key\" {print $10}'"
proc = subprocess.Popen([cmdStr],
stdout=subprocess.PIPE, shell=True)
(result, err) = proc.communicate()
if err:
return None
if not result:
return None
if len(result) < 5:
return None
return result.decode('utf-8').replace('"', '').strip()
def _pgpLocalPublicKey() -> str:
"""Gets the local pgp public key
"""
keyId = _pgpLocalPublicKeyId()
if not keyId:
return None
cmdStr = "gpg --armor --export " + keyId
proc = subprocess.Popen([cmdStr],
stdout=subprocess.PIPE, shell=True)
(result, err) = proc.communicate()
if err:
return None
if not result:
return None
return extractPGPPublicKey(result.decode('utf-8'))
def pgpPublicKeyUpload(baseDir: str, session,
nickname: str, password: str,
domain: str, port: int,
httpPrefix: str,
cachedWebfingers: {}, personCache: {},
debug: bool, test: str) -> {}:
if debug:
print('pgpPublicKeyUpload')
if not session:
if debug:
print('WARN: No session for pgpPublicKeyUpload')
return None
if not test:
if debug:
print('Getting PGP public key')
PGPpubKey = _pgpLocalPublicKey()
if not PGPpubKey:
return None
PGPpubKeyId = _pgpLocalPublicKeyId()
else:
if debug:
print('Testing with PGP public key ' + test)
PGPpubKey = test
PGPpubKeyId = None
domainFull = getFullDomain(domain, port)
if debug:
print('PGP test domain: ' + domainFull)
handle = nickname + '@' + domainFull
if debug:
print('Getting actor for ' + handle)
actorJson = getActorJson(handle, False, False, debug, True)
if not actorJson:
if debug:
print('No actor returned for ' + handle)
return None
if debug:
print('Actor for ' + handle + ' obtained')
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
handle = actor.replace('/users/', '/@')
# check that this looks like the correct actor
if not actorJson.get('id'):
if debug:
print('Actor has no id')
return None
if not actorJson.get('url'):
if debug:
print('Actor has no url')
return None
if not actorJson.get('type'):
if debug:
print('Actor has no type')
return None
if actorJson['id'] != actor:
if debug:
print('Actor id is not ' + actor +
' instead is ' + actorJson['id'])
return None
if actorJson['url'] != handle:
if debug:
print('Actor url is not ' + handle)
return None
if actorJson['type'] != 'Person':
if debug:
print('Actor type is not Person')
return None
# set the pgp details
if PGPpubKeyId:
setPGPfingerprint(actorJson, PGPpubKeyId)
else:
if debug:
print('No PGP key Id. Continuing anyway.')
if debug:
print('Setting PGP key within ' + actor)
setPGPpubKey(actorJson, PGPpubKey)
# create an actor update
statusNumber, published = getStatusNumber()
actorUpdate = {
'@context': 'https://www.w3.org/ns/activitystreams',
'id': actor + '#updates/' + statusNumber,
'type': 'Update',
'actor': actor,
'to': [actor],
'cc': [],
'object': actorJson
}
if debug:
print('actor update is ' + str(actorUpdate))
# lookup the inbox for the To handle
wfRequest = \
webfingerHandle(session, handle, httpPrefix, cachedWebfingers,
domain, __version__, debug)
if not wfRequest:
if debug:
print('DEBUG: pgp actor update webfinger failed for ' +
handle)
return None
if not isinstance(wfRequest, dict):
if debug:
print('WARN: Webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return None
postToBox = 'outbox'
# get the actor inbox for the To handle
(inboxUrl, pubKeyId, pubKey,
fromPersonId, sharedInbox, avatarUrl,
displayName) = getPersonBox(baseDir, session, wfRequest, personCache,
__version__, httpPrefix, nickname,
domain, postToBox, 52025)
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
return None
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
return None
authHeader = createBasicAuthHeader(nickname, password)
headers = {
'host': domain,
'Content-type': 'application/json',
'Authorization': authHeader
}
quiet = not debug
tries = 0
while tries < 4:
postResult = \
postJson(session, actorUpdate, [], inboxUrl,
headers, 5, quiet)
if postResult:
break
tries += 1
if postResult is None:
if debug:
print('DEBUG: POST pgp actor update failed for c2s to ' +
inboxUrl)
return None
if debug:
print('DEBUG: c2s POST pgp actor update success')
return actorUpdate

369
posts.py
View File

@ -40,8 +40,6 @@ from utils import validPostDate
from utils import getFullDomain
from utils import getFollowersList
from utils import isEvil
from utils import removeIdEnding
from utils import getCachedPostFilename
from utils import getStatusNumber
from utils import createPersonDir
from utils import urlPermitted
@ -259,7 +257,7 @@ def getPersonBox(baseDir: str, session, wfRequest: {},
personJson = getJson(session, personUrl, asHeader, None,
debug, projectVersion, httpPrefix, domain)
if not personJson:
print('Unable to get actor')
print('Unable to get actor for ' + personUrl)
return None, None, None, None, None, None, None
boxJson = None
if not personJson.get(boxName):
@ -1827,17 +1825,6 @@ def createReportPost(baseDir: str,
if not postJsonObject:
continue
# update the inbox index with the report filename
# indexFilename = baseDir+'/accounts/'+handle+'/inbox.index'
# indexEntry = \
# removeIdEnding(postJsonObject['id']).replace('/','#') + '.json'
# if indexEntry not in open(indexFilename).read():
# try:
# with open(indexFilename, 'a+') as fp:
# fp.write(indexEntry)
# except:
# pass
# save a notification file so that the moderator
# knows something new has appeared
newReportFile = baseDir + '/accounts/' + handle + '/.newReport'
@ -2056,11 +2043,11 @@ def sendPostViaServer(projectVersion: str,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: webfinger failed for ' + handle)
print('DEBUG: post webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: post webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -2078,11 +2065,12 @@ def sendPostViaServer(projectVersion: str,
82796)
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: post no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: post no actor was found for ' + handle)
return 4
# Get the json for the c2s post, not saving anything to file
@ -2131,7 +2119,7 @@ def sendPostViaServer(projectVersion: str,
inboxUrl, headers)
if not postResult:
if debug:
print('DEBUG: Failed to upload image')
print('DEBUG: post failed to upload image')
# return 9
headers = {
@ -2142,7 +2130,7 @@ def sendPostViaServer(projectVersion: str,
postDumps = json.dumps(postJsonObject)
postResult = \
postJsonString(session, postDumps, [],
inboxUrl, headers, debug, 60, True)
inboxUrl, headers, debug, 5, True)
if not postResult:
if debug:
print('DEBUG: POST failed for c2s to ' + inboxUrl)
@ -2364,7 +2352,7 @@ def sendSignedJson(postJsonObject: {}, session, baseDir: str,
def addToField(activityType: str, postJsonObject: {},
debug: bool) -> ({}, bool):
"""The Follow activity doesn't have a 'to' field and so one
"""The Follow/Add/Remove activity doesn't have a 'to' field and so one
needs to be added so that activity distribution happens in a consistent way
Returns true if a 'to' field exists or was added
"""
@ -2383,19 +2371,34 @@ def addToField(activityType: str, postJsonObject: {},
if postJsonObject['type'] == activityType:
isSameType = True
if debug:
print('DEBUG: "to" field assigned to Follow')
print('DEBUG: "to" field assigned to ' + activityType)
toAddress = postJsonObject['object']
if '/statuses/' in toAddress:
toAddress = toAddress.split('/statuses/')[0]
postJsonObject['to'] = [toAddress]
toFieldAdded = True
elif isinstance(postJsonObject['object'], dict):
if postJsonObject['object'].get('type'):
# add a to field to bookmark add or remove
if postJsonObject.get('type') and \
postJsonObject.get('actor') and \
postJsonObject['object'].get('type'):
if postJsonObject['type'] == 'Add' or \
postJsonObject['type'] == 'Remove':
if postJsonObject['object']['type'] == 'Document':
postJsonObject['to'] = \
[postJsonObject['actor']]
postJsonObject['object']['to'] = \
[postJsonObject['actor']]
toFieldAdded = True
if not toFieldAdded and \
postJsonObject['object'].get('type'):
if postJsonObject['object']['type'] == activityType:
isSameType = True
if isinstance(postJsonObject['object']['object'], str):
if debug:
print('DEBUG: "to" field assigned to Follow')
print('DEBUG: "to" field assigned to ' +
activityType)
toAddress = postJsonObject['object']['object']
if '/statuses/' in toAddress:
toAddress = toAddress.split('/statuses/')[0]
@ -2426,8 +2429,8 @@ def sendToNamedAddresses(session, baseDir: str,
return
if not postJsonObject.get('object'):
return
if isinstance(postJsonObject['object'], dict):
isProfileUpdate = False
if isinstance(postJsonObject['object'], dict):
# for actor updates there is no 'to' within the object
if postJsonObject['object'].get('type') and postJsonObject.get('type'):
if (postJsonObject['type'] == 'Update' and
@ -2510,6 +2513,16 @@ def sendToNamedAddresses(session, baseDir: str,
toDomain, toPort = getDomainFromActor(address)
if not toDomain:
continue
# Don't send profile/actor updates to yourself
if isProfileUpdate:
domainFull = getFullDomain(domain, port)
toDomainFull = getFullDomain(toDomain, toPort)
if nickname == toNickname and \
domainFull == toDomainFull:
if debug:
print('Not sending profile update to self. ' +
nickname + '@' + domainFull)
continue
if debug:
domainFull = getFullDomain(domain, port)
toDomainFull = getFullDomain(toDomain, toPort)
@ -3245,7 +3258,7 @@ def _createBoxIndexed(recentPostsCache: {},
# created by individualPostAsHtml
p['hasReplies'] = hasReplies
# Don't show likes, replies, DMs or shares (announces) to
# Don't show likes, replies, bookmarks, DMs or shares (announces) to
# unauthorized viewers
if not authorized:
if p.get('object'):
@ -3260,6 +3273,8 @@ def _createBoxIndexed(recentPostsCache: {},
p['shares'] = {}
if p['object'].get('bookmarks'):
p['bookmarks'] = {}
if p['object'].get('ignores'):
p['ignores'] = {}
boxItems['orderedItems'].append(p)
@ -4039,87 +4054,6 @@ def isMuted(baseDir: str, nickname: str, domain: str, postId: str) -> bool:
return False
def mutePost(baseDir: str, nickname: str, domain: str, postId: str,
recentPostsCache: {}) -> None:
""" Mutes the given post
"""
postFilename = locatePost(baseDir, nickname, domain, postId)
if not postFilename:
return
postJsonObject = loadJson(postFilename)
if not postJsonObject:
return
# remove cached post so that the muted version gets recreated
# without its content text and/or image
cachedPostFilename = \
getCachedPostFilename(baseDir, nickname, domain, postJsonObject)
if cachedPostFilename:
if os.path.isfile(cachedPostFilename):
os.remove(cachedPostFilename)
muteFile = open(postFilename + '.muted', 'w+')
if muteFile:
muteFile.write('\n')
muteFile.close()
print('MUTE: ' + postFilename + '.muted file added')
# if the post is in the recent posts cache then mark it as muted
if recentPostsCache.get('index'):
postId = \
removeIdEnding(postJsonObject['id']).replace('/', '#')
if postId in recentPostsCache['index']:
print('MUTE: ' + postId + ' is in recent posts cache')
if recentPostsCache['json'].get(postId):
postJsonObject['muted'] = True
recentPostsCache['json'][postId] = json.dumps(postJsonObject)
if recentPostsCache.get('html'):
if recentPostsCache['html'].get(postId):
del recentPostsCache['html'][postId]
print('MUTE: ' + postId +
' marked as muted in recent posts memory cache')
def unmutePost(baseDir: str, nickname: str, domain: str, postId: str,
recentPostsCache: {}) -> None:
""" Unmutes the given post
"""
postFilename = locatePost(baseDir, nickname, domain, postId)
if not postFilename:
return
postJsonObject = loadJson(postFilename)
if not postJsonObject:
return
muteFilename = postFilename + '.muted'
if os.path.isfile(muteFilename):
os.remove(muteFilename)
print('UNMUTE: ' + muteFilename + ' file removed')
# remove cached post so that the muted version gets recreated
# with its content text and/or image
cachedPostFilename = \
getCachedPostFilename(baseDir, nickname, domain, postJsonObject)
if cachedPostFilename:
if os.path.isfile(cachedPostFilename):
os.remove(cachedPostFilename)
# if the post is in the recent posts cache then mark it as unmuted
if recentPostsCache.get('index'):
postId = \
removeIdEnding(postJsonObject['id']).replace('/', '#')
if postId in recentPostsCache['index']:
print('UNMUTE: ' + postId + ' is in recent posts cache')
if recentPostsCache['json'].get(postId):
postJsonObject['muted'] = False
recentPostsCache['json'][postId] = json.dumps(postJsonObject)
if recentPostsCache.get('html'):
if recentPostsCache['html'].get(postId):
del recentPostsCache['html'][postId]
print('UNMUTE: ' + postId +
' marked as unmuted in recent posts cache')
def sendBlockViaServer(baseDir: str, session,
fromNickname: str, password: str,
fromDomain: str, fromPort: int,
@ -4156,11 +4090,11 @@ def sendBlockViaServer(baseDir: str, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: block webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: block Webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -4175,11 +4109,11 @@ def sendBlockViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: block no ' + postToBox + ' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: block no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -4192,7 +4126,7 @@ def sendBlockViaServer(baseDir: str, session,
postResult = postJson(session, newBlockJson, [], inboxUrl,
headers, 30, True)
if not postResult:
print('WARN: Unable to post block')
print('WARN: block unable to post')
if debug:
print('DEBUG: c2s POST block success')
@ -4200,6 +4134,162 @@ def sendBlockViaServer(baseDir: str, session,
return newBlockJson
def sendMuteViaServer(baseDir: str, session,
fromNickname: str, password: str,
fromDomain: str, fromPort: int,
httpPrefix: str, mutedUrl: str,
cachedWebfingers: {}, personCache: {},
debug: bool, projectVersion: str) -> {}:
"""Creates a mute via c2s
"""
if not session:
print('WARN: No session for sendMuteViaServer')
return 6
fromDomainFull = getFullDomain(fromDomain, fromPort)
actor = httpPrefix + '://' + fromDomainFull + '/users/' + fromNickname
handle = actor.replace('/users/', '/@')
newMuteJson = {
"@context": "https://www.w3.org/ns/activitystreams",
'type': 'Ignore',
'actor': actor,
'to': [actor],
'object': mutedUrl
}
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: mute webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: mute Webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
# get the actor inbox for the To handle
(inboxUrl, pubKeyId, pubKey,
fromPersonId, sharedInbox, avatarUrl,
displayName) = getPersonBox(baseDir, session, wfRequest,
personCache,
projectVersion, httpPrefix, fromNickname,
fromDomain, postToBox, 72652)
if not inboxUrl:
if debug:
print('DEBUG: mute no ' + postToBox + ' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: mute no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
headers = {
'host': fromDomain,
'Content-type': 'application/json',
'Authorization': authHeader
}
postResult = postJson(session, newMuteJson, [], inboxUrl,
headers, 3, True)
if postResult is None:
print('WARN: mute unable to post')
if debug:
print('DEBUG: c2s POST mute success')
return newMuteJson
def sendUndoMuteViaServer(baseDir: str, session,
fromNickname: str, password: str,
fromDomain: str, fromPort: int,
httpPrefix: str, mutedUrl: str,
cachedWebfingers: {}, personCache: {},
debug: bool, projectVersion: str) -> {}:
"""Undoes a mute via c2s
"""
if not session:
print('WARN: No session for sendUndoMuteViaServer')
return 6
fromDomainFull = getFullDomain(fromDomain, fromPort)
actor = httpPrefix + '://' + fromDomainFull + '/users/' + fromNickname
handle = actor.replace('/users/', '/@')
undoMuteJson = {
"@context": "https://www.w3.org/ns/activitystreams",
'type': 'Undo',
'actor': actor,
'to': [actor],
'object': {
'type': 'Ignore',
'actor': actor,
'to': [actor],
'object': mutedUrl
}
}
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: undo mute webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: undo mute Webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
# get the actor inbox for the To handle
(inboxUrl, pubKeyId, pubKey,
fromPersonId, sharedInbox, avatarUrl,
displayName) = getPersonBox(baseDir, session, wfRequest,
personCache,
projectVersion, httpPrefix, fromNickname,
fromDomain, postToBox, 72652)
if not inboxUrl:
if debug:
print('DEBUG: undo mute no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: undo mute no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
headers = {
'host': fromDomain,
'Content-type': 'application/json',
'Authorization': authHeader
}
postResult = postJson(session, undoMuteJson, [], inboxUrl,
headers, 3, True)
if postResult is None:
print('WARN: undo mute unable to post')
if debug:
print('DEBUG: c2s POST undo mute success')
return undoMuteJson
def sendUndoBlockViaServer(baseDir: str, session,
fromNickname: str, password: str,
fromDomain: str, fromPort: int,
@ -4240,11 +4330,11 @@ def sendUndoBlockViaServer(baseDir: str, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: unblock webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: unblock webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -4258,11 +4348,12 @@ def sendUndoBlockViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: unblock no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: unblock no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -4275,10 +4366,10 @@ def sendUndoBlockViaServer(baseDir: str, session,
postResult = postJson(session, newBlockJson, [], inboxUrl,
headers, 30, True)
if not postResult:
print('WARN: Unable to post block')
print('WARN: unblock unable to post')
if debug:
print('DEBUG: c2s POST block success')
print('DEBUG: c2s POST unblock success')
return newBlockJson
@ -4305,3 +4396,39 @@ def postIsMuted(baseDir: str, nickname: str, domain: str,
if os.path.isfile(muteFilename):
return True
return False
def c2sBoxJson(baseDir: str, session,
nickname: str, password: str,
domain: str, port: int,
httpPrefix: str,
boxName: str, pageNumber: int,
debug: bool) -> {}:
"""C2S Authenticated GET of posts for a timeline
"""
if not session:
print('WARN: No session for c2sBoxJson')
return None
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
authHeader = createBasicAuthHeader(nickname, password)
profileStr = 'https://www.w3.org/ns/activitystreams'
headers = {
'host': domain,
'Content-type': 'application/json',
'Authorization': authHeader,
'Accept': 'application/ld+json; profile="' + profileStr + '"'
}
# GET json
url = actor + '/' + boxName + '?page=' + str(pageNumber)
boxJson = getJson(session, url, headers, None,
debug, __version__, httpPrefix, None)
if boxJson is not None and debug:
print('DEBUG: GET c2sBoxJson success')
return boxJson

13
roles.py
View File

@ -313,11 +313,11 @@ def sendRoleViaServer(baseDir: str, session,
delegatorDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: role webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: role webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -334,11 +334,12 @@ def sendRoleViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: role no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: role no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(delegatorNickname, password)
@ -352,7 +353,7 @@ def sendRoleViaServer(baseDir: str, session,
postJson(session, newRoleJson, [], inboxUrl, headers, 30, True)
if not postResult:
if debug:
print('DEBUG: POST announce failed for c2s to ' + inboxUrl)
print('DEBUG: POST role failed for c2s to ' + inboxUrl)
# return 5
if debug:

6
session.py
View File

@ -152,6 +152,12 @@ def postJson(session, postJsonObject: {}, federationList: [],
session.post(url=inboxUrl,
data=json.dumps(postJsonObject),
headers=headers, timeout=timeoutSec)
except requests.Timeout as e:
if not quiet:
print('ERROR: postJson timeout ' + inboxUrl + ' ' +
json.dumps(postJsonObject) + ' ' + str(headers))
print(e)
return ''
except requests.exceptions.RequestException as e:
if not quiet:
print('ERROR: postJson requests failed ' + inboxUrl + ' ' +

28
shares.py
View File

@ -361,11 +361,11 @@ def sendShareViaServer(baseDir, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: share webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: share webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -381,11 +381,12 @@ def sendShareViaServer(baseDir, session,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: share no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: share no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -409,7 +410,7 @@ def sendShareViaServer(baseDir, session,
postJson(session, newShareJson, [], inboxUrl, headers, 30, True)
if not postResult:
if debug:
print('DEBUG: POST announce failed for c2s to ' + inboxUrl)
print('DEBUG: POST share failed for c2s to ' + inboxUrl)
# return 5
if debug:
@ -460,11 +461,11 @@ def sendUndoShareViaServer(baseDir: str, session,
fromDomain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: unshare webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: unshare webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -480,11 +481,12 @@ def sendUndoShareViaServer(baseDir: str, session,
if not inboxUrl:
if debug:
print('DEBUG: No '+postToBox+' was found for ' + handle)
print('DEBUG: unshare no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: unshare no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(fromNickname, password)
@ -499,11 +501,11 @@ def sendUndoShareViaServer(baseDir: str, session,
headers, 30, True)
if not postResult:
if debug:
print('DEBUG: POST announce failed for c2s to ' + inboxUrl)
print('DEBUG: POST unshare failed for c2s to ' + inboxUrl)
# return 5
if debug:
print('DEBUG: c2s POST undo share success')
print('DEBUG: c2s POST unshare success')
return undoShareJson

13
skills.py
View File

@ -126,11 +126,11 @@ def sendSkillViaServer(baseDir: str, session, nickname: str, password: str,
domain, projectVersion, debug)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
print('DEBUG: skill webfinger failed for ' + handle)
return 1
if not isinstance(wfRequest, dict):
print('WARN: Webfinger for ' + handle + ' did not return a dict. ' +
str(wfRequest))
print('WARN: skill webfinger for ' + handle +
' did not return a dict. ' + str(wfRequest))
return 1
postToBox = 'outbox'
@ -145,11 +145,12 @@ def sendSkillViaServer(baseDir: str, session, nickname: str, password: str,
if not inboxUrl:
if debug:
print('DEBUG: No ' + postToBox + ' was found for ' + handle)
print('DEBUG: skill no ' + postToBox +
' was found for ' + handle)
return 3
if not fromPersonId:
if debug:
print('DEBUG: No actor was found for ' + handle)
print('DEBUG: skill no actor was found for ' + handle)
return 4
authHeader = createBasicAuthHeader(nickname, password)
@ -164,7 +165,7 @@ def sendSkillViaServer(baseDir: str, session, nickname: str, password: str,
headers, 30, True)
if not postResult:
if debug:
print('DEBUG: POST announce failed for c2s to ' + inboxUrl)
print('DEBUG: POST skill failed for c2s to ' + inboxUrl)
# return 5
if debug:

61
speaker.py
View File

@ -10,8 +10,6 @@ import os
import html
import random
import urllib.parse
from auth import createBasicAuthHeader
from session import getJson
from utils import isDM
from utils import isReply
from utils import camelCaseSplit
@ -22,7 +20,6 @@ from utils import getDisplayName
from utils import removeHtml
from utils import loadJson
from utils import saveJson
from utils import getFullDomain
from utils import isPGPEncrypted
from content import htmlReplaceQuoteMarks
@ -168,8 +165,10 @@ def speakerReplaceLinks(sayText: str, translate: {},
Instead of reading out potentially very long and meaningless links
"""
text = sayText
text = text.replace('?v=', '__v=')
for ch in speakerRemoveChars:
text = text.replace(ch, ' ')
text = text.replace('__v=', '?v=')
replacements = {}
wordsList = text.split(' ')
if translate.get('Linked'):
@ -253,38 +252,6 @@ def _removeEmojiFromText(sayText: str) -> str:
return sayText.replace(' ', ' ').strip()
def getSpeakerFromServer(baseDir: str, session,
nickname: str, password: str,
domain: str, port: int,
httpPrefix: str,
debug: bool, projectVersion: str) -> {}:
"""Returns some json which contains the latest inbox
entry in a minimal format suitable for a text-to-speech reader
"""
if not session:
print('WARN: No session for getSpeakerFromServer')
return 6
domainFull = getFullDomain(domain, port)
authHeader = createBasicAuthHeader(nickname, password)
headers = {
'host': domain,
'Content-type': 'application/json',
'Authorization': authHeader
}
url = \
httpPrefix + '://' + \
domainFull + '/users/' + nickname + '/speaker'
speakerJson = \
getJson(session, url, headers, None, debug,
__version__, httpPrefix, domain, 20, True)
return speakerJson
def _speakerEndpointJson(displayName: str, summary: str,
content: str, sayContent: str,
imageDescription: str,
@ -405,6 +372,30 @@ def getSSMLbox(baseDir: str, path: str,
instanceTitle, gender)
def speakableText(baseDir: str, content: str, translate: {}) -> (str, []):
"""Convert the given text to a speakable version
which includes changes for prononciation
"""
if isPGPEncrypted(content):
return content, []
# replace some emoji before removing html
if ' <3' in content:
content = content.replace(' <3', ' ' + translate['heart'])
content = removeHtml(htmlReplaceQuoteMarks(content))
detectedLinks = []
content = speakerReplaceLinks(content, translate, detectedLinks)
# replace all double spaces
while ' ' in content:
content = content.replace(' ', ' ')
content = content.replace(' . ', '. ').strip()
sayContent = _speakerPronounce(baseDir, content, translate)
# replace all double spaces
while ' ' in sayContent:
sayContent = sayContent.replace(' ', ' ')
return sayContent.replace(' . ', '. ').strip(), detectedLinks
def _postToSpeakerJson(baseDir: str, httpPrefix: str,
nickname: str, domain: str, domainFull: str,
postJsonObject: {}, personCache: {},

137
tests.py
View File

@ -53,6 +53,7 @@ from utils import getFollowersOfPerson
from utils import removeHtml
from utils import dangerousMarkup
from pgp import extractPGPPublicKey
from pgp import pgpPublicKeyUpload
from utils import containsPGPPublicKey
from follow import followerOfPerson
from follow import unfollowAccount
@ -1574,7 +1575,7 @@ def testClientToServer():
sessionAlice = createSession(proxyType)
followersOnly = False
attachedImageFilename = baseDir+'/img/logo.png'
attachedImageFilename = baseDir + '/img/logo.png'
mediaType = getAttachmentMediaType(attachedImageFilename)
attachedImageDescription = 'Logo'
isArticle = False
@ -1910,6 +1911,22 @@ def testActorParsing():
def testWebLinks():
print('testWebLinks')
exampleText = \
"<p>Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + \
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + \
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + \
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + \
" <a href=\"https://domain.ugh/tags/turbot\" class=\"mention " + \
"hashtag\" rel=\"tag\">#<span>turbot</span></a> <a href=\"" + \
"https://domain.ugh/tags/haddock\" class=\"mention hashtag\"" + \
" rel=\"tag\">#<span>haddock</span></a></p>"
resultText = removeLongWords(exampleText, 40, [])
assert resultText == "<p>Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + \
" <a href=\"https://domain.ugh/tags/turbot\" class=\"mention " + \
"hashtag\" rel=\"tag\">#<span>turbot</span></a> " + \
"<a href=\"https://domain.ugh/tags/haddock\" " + \
"class=\"mention hashtag\" rel=\"tag\">#<span>haddock</span></a></p>"
exampleText = \
'<p><span class=\"h-card\"><a href=\"https://something/@orother' + \
'\" class=\"u-url mention\">@<span>foo</span></a></span> Some ' + \
@ -2701,6 +2718,8 @@ def testFirstParagraphFromString():
'<p><a href="https://somesite.com/somepath">This is a test</a></p>' + \
'<p>This is another paragraph</p>'
resultStr = firstParagraphFromString(testStr)
if resultStr != 'This is a test':
print(resultStr)
assert resultStr == 'This is a test'
testStr = 'Testing without html'
@ -3447,6 +3466,122 @@ def testExtractPGPPublicKey():
assert result == pubKey
def testUpdateActor():
print('Testing update of actor properties')
global testServerAliceRunning
testServerAliceRunning = False
httpPrefix = 'http'
proxyType = None
federationList = []
baseDir = os.getcwd()
if os.path.isdir(baseDir + '/.tests'):
shutil.rmtree(baseDir + '/.tests')
os.mkdir(baseDir + '/.tests')
# create the server
aliceDir = baseDir + '/.tests/alice'
aliceDomain = '127.0.0.11'
alicePort = 61792
aliceSendThreads = []
bobAddress = '127.0.0.84:6384'
global thrAlice
if thrAlice:
while thrAlice.is_alive():
thrAlice.stop()
time.sleep(1)
thrAlice.kill()
thrAlice = \
threadWithTrace(target=createServerAlice,
args=(aliceDir, aliceDomain, alicePort, bobAddress,
federationList, False, False,
aliceSendThreads),
daemon=True)
thrAlice.start()
assert thrAlice.is_alive() is True
# wait for server to be running
ctr = 0
while not testServerAliceRunning:
time.sleep(1)
ctr += 1
if ctr > 60:
break
print('Alice online: ' + str(testServerAliceRunning))
print('\n\n*******************************************************')
print('Alice updates her PGP key')
sessionAlice = createSession(proxyType)
cachedWebfingers = {}
personCache = {}
password = 'alicepass'
outboxPath = aliceDir + '/accounts/alice@' + aliceDomain + '/outbox'
actorFilename = aliceDir + '/accounts/' + 'alice@' + aliceDomain + '.json'
assert os.path.isfile(actorFilename)
assert len([name for name in os.listdir(outboxPath)
if os.path.isfile(os.path.join(outboxPath, name))]) == 0
pubKey = \
'-----BEGIN PGP PUBLIC KEY BLOCK-----\n\n' + \
'mDMEWZBueBYJKwYBBAHaRw8BAQdAKx1t6wL0RTuU6/' + \
'IBjngMbVJJ3Wg/3UW73/PV\n' + \
'I47xKTS0IUJvYiBNb3R0cmFtIDxib2JAZnJlZWRvb' + \
'WJvbmUubmV0PoiQBBMWCAA4\n' + \
'FiEEmruCwAq/OfgmgEh9zCU2GR+nwz8FAlmQbngCG' + \
'wMFCwkIBwMFFQoJCAsFFgID\n' + \
'AQACHgECF4AACgkQzCU2GR+nwz/9sAD/YgsHnVszH' + \
'Nz1zlVc5EgY1ByDupiJpHj0\n' + \
'XsLYk3AbNRgBALn45RqgD4eWHpmOriH09H5Rc5V9i' + \
'N4+OiGUn2AzJ6oHuDgEWZBu\n' + \
'eBIKKwYBBAGXVQEFAQEHQPRBG2ZQJce475S3e0Dxe' + \
'b0Fz5WdEu2q3GYLo4QG+4Ry\n' + \
'AwEIB4h4BBgWCAAgFiEEmruCwAq/OfgmgEh9zCU2G' + \
'R+nwz8FAlmQbngCGwwACgkQ\n' + \
'zCU2GR+nwz+OswD+JOoyBku9FzuWoVoOevU2HH+bP' + \
'OMDgY2OLnST9ZSyHkMBAMcK\n' + \
'fnaZ2Wi050483Sj2RmQRpb99Dod7rVZTDtCqXk0J\n' + \
'=gv5G\n' + \
'-----END PGP PUBLIC KEY BLOCK-----'
actorUpdate = \
pgpPublicKeyUpload(aliceDir, sessionAlice,
'alice', password,
aliceDomain, alicePort,
httpPrefix,
cachedWebfingers, personCache,
True, pubKey)
print('actor update result: ' + str(actorUpdate))
assert actorUpdate
# load alice actor
print('Loading actor: ' + actorFilename)
actorJson = loadJson(actorFilename)
assert actorJson
if len(actorJson['attachment']) == 0:
print("actorJson['attachment'] has no contents")
assert len(actorJson['attachment']) > 0
propertyFound = False
for propertyValue in actorJson['attachment']:
if propertyValue['name'] == 'PGP':
print('PGP property set within attachment')
assert pubKey in propertyValue['value']
propertyFound = True
assert propertyFound
# stop the server
thrAlice.kill()
thrAlice.join()
assert thrAlice.is_alive() is False
os.chdir(baseDir)
if os.path.isdir(baseDir + '/.tests'):
shutil.rmtree(baseDir + '/.tests')
def runAllTests():
print('Running tests...')
testFunctions()

34
utils.py
View File

@ -13,6 +13,7 @@ import shutil
import datetime
import json
import idna
import locale
from pprint import pprint
from calendar import monthrange
from followingCalendar import addPersonToCalendar
@ -252,6 +253,7 @@ def removeHtml(content: str) -> str:
if '<' not in content:
return content
removing = False
content = content.replace('<a href', ' <a href')
content = content.replace('<q>', '"').replace('</q>', '"')
result = ''
for ch in content:
@ -261,6 +263,7 @@ def removeHtml(content: str) -> str:
removing = False
elif not removing:
result += ch
result = result.replace(' ', ' ').strip()
return result
@ -1357,7 +1360,9 @@ def _isReservedName(nickname: str) -> bool:
'accounts', 'channels', 'profile', 'u',
'updates', 'repeat', 'announce',
'shares', 'fonts', 'icons', 'avatars',
'welcome', 'helpimages')
'welcome', 'helpimages',
'bookmark', 'bookmarks', 'tlbookmarks',
'ignores')
if nickname in reservedNames:
return True
return False
@ -2150,3 +2155,30 @@ def isPGPEncrypted(content: str) -> bool:
if '--END PGP MESSAGE--' in content:
return True
return False
def loadTranslationsFromFile(baseDir: str, language: str) -> ({}, str):
"""Returns the translations dictionary
"""
if not os.path.isdir(baseDir + '/translations'):
print('ERROR: translations directory not found')
return
if not language:
systemLanguage = locale.getdefaultlocale()[0]
else:
systemLanguage = language
if not systemLanguage:
systemLanguage = 'en'
if '_' in systemLanguage:
systemLanguage = systemLanguage.split('_')[0]
while '/' in systemLanguage:
systemLanguage = systemLanguage.split('/')[1]
if '.' in systemLanguage:
systemLanguage = systemLanguage.split('.')[0]
translationsFile = baseDir + '/translations/' + \
systemLanguage + '.json'
if not os.path.isfile(translationsFile):
systemLanguage = 'en'
translationsFile = baseDir + '/translations/' + \
systemLanguage + '.json'
return loadJson(translationsFile), systemLanguage