Merge branch 'main' of gitlab.com:bashrc2/epicyon

merge-requests/30/head
Bob Mottram 2021-08-16 11:32:55 +01:00
commit b93d2565db
224 changed files with 45448 additions and 2933 deletions

View File

@ -1,4 +1,4 @@
FROM debian:buster-slim
FROM debian:bullseye-slim
ENV DOMAIN=localhost
RUN apt-get update && \
apt-get -y install \

View File

@ -5,6 +5,8 @@ all:
debug:
source:
rm -f *.*~ *~
rm -f ontology/*~
rm -f ontology/*.new
rm -f translations/*~
rm -f orgs/*~
rm -f scripts/*~
@ -17,6 +19,8 @@ source:
clean:
rm -f *.*~ *~ *.dot
rm -f orgs/*~
rm -f ontology/*~
rm -f ontology/*.new
rm -f defaultwelcome/*~
rm -f theme/indymediaclassic/welcome/*~
rm -f theme/indymediamodern/welcome/*~

View File

@ -4,9 +4,9 @@ Add issues on https://gitlab.com/bashrc2/epicyon/-/issues
<blockquote><b>Epicyon</b>, meaning <i>"more than a dog"</i>. Largest of the <i>Borophaginae</i> which lived in North America 20-5 million years ago.</blockquote>
<img src="https://epicyon.net/img/screenshot_starlight.jpg" width="80%"/>
<img src="https://libreserver.org/epicyon/img/screenshot_starlight.jpg" width="80%"/>
<img src="https://epicyon.net/img/mobile.jpg" width="30%"/>
<img src="https://libreserver.org/epicyon/img/mobile.jpg" width="30%"/>
Epicyon is a modern [ActivityPub](https://www.w3.org/TR/activitypub) compliant server implementing both S2S and C2S protocols and suitable for installation on single board computers. It includes features such as moderation tools, post expiry, content warnings, image descriptions, news feed and perimeter defense against adversaries. It contains *no JavaScript* and uses HTML+CSS with a Python backend.
@ -16,9 +16,9 @@ Matrix room: **#epicyon:matrix.freedombone.net**
Includes emojis designed by [OpenMoji](https://openmoji.org) the open-source emoji and icon project. License: [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0). Blob Cat Emoji and Meowmoji were made by Nitro Blob Hub, licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). [Digital Pets emoji](https://opengameart.org/content/16x16-emotes-for-rpgs-and-digital-pets) were made by Tomcat94 and licensed under CC0.
<img src="https://epicyon.net/img/screenshot_light.jpg" width="80%"/>
<img src="https://libreserver.org/epicyon/img/screenshot_light.jpg" width="80%"/>
<img src="https://epicyon.net/img/screenshot_login.jpg" width="80%"/>
<img src="https://libreserver.org/epicyon/img/screenshot_login.jpg" width="80%"/>
## Package Dependencies

View File

@ -34,19 +34,23 @@
* Integration with RSS feeds, for reading news or blogs
* Moderation capabilities for posts, hashtags and blocks
**Features which won't be implemented**
## Non-goals
The following are considered anti-features of other social network systems, since they encourage dysfunctional social interactions.
* Features designed to scale to large numbers of accounts (say, more than 20 active users)
* Trending hashtags, or trending anything
* Ranking, rating or recommending mechanisms for posts or people (other than likes or repeats/boosts)
* Geo-location features
* Geo-location features, unless they're always opt-in
* Algorithmic timelines (i.e. non-chronological)
* Direct payment mechanisms, although integration with other services may be possible
* Any variety of blockchain
* Anything based upon "proof of stake". The "people who have more, get more" principle should be rejected.
* Like counts above some small maximum number. The aim is to avoid people getting addicted to making numbers go up, and especially to avoid the dark market in fake likes.
* Sponsored posts
* Enterprise features for use cases applicable only to businesses. Epicyon could be used in a small business, but it's not primarily designed for that
* Collaborative editing of posts, although you could do that outside of this system using Etherpad, or similar
* Anonymous posts from random internet users published under a single generic instance account
* Hierarchies of roles beyond ordinary moderation, such as X requires special agreement from Y before sending a post
* Hierarchies of roles beyond ordinary moderation, such as X requires special agreement from Y before sending a post. Originally delegated roles were envisioned, but later abandoned due to the potential for creating elaborate hierarchies
* Federated blocklists. Initially this seems like a good idea, but the potential down sides outweigh the benefits. eg. Two allied instances share their global blocklist. Some time later one instance is transferred to an adversary, or gets hacked or sold. Adversary can now control your global blocklist and trash your instance very quickly that way.
* Federated moderation. Again, seems like it might be beneficial initially. Share the burden of moderation. But under realistic conditions people could be pressured or bribed into giving federated moderation access, and the consequences could be very bad. Individuals going on power trips, controlling multiple instances and heading back towards centralization. Avoid creating technical routes which easily lead to power consolidation and centralization.

View File

@ -6,8 +6,9 @@
## Groups
* Unit test for group creation
* Groups can be defined as having particular roles/skills
* Parse posts from Lemmy groups
* Think of a way to display groups. Maybe assign a hashtag and display them like hashtag timelines
## Questions
@ -19,6 +20,7 @@
## Code
* More unit test coverage
* Unit test for federated shared items
* Break up large functions into smaller ones
* Architecture diagrams
* Code documentation?

View File

@ -17,6 +17,8 @@ from utils import domainPermitted
from utils import followPerson
from utils import hasObjectDict
from utils import acctDir
from utils import hasGroupType
from utils import localActorUrl
def _createAcceptReject(baseDir: str, federationList: [],
@ -40,7 +42,7 @@ def _createAcceptReject(baseDir: str, federationList: [],
newAccept = {
"@context": "https://www.w3.org/ns/activitystreams",
'type': acceptType,
'actor': httpPrefix + '://' + domain + '/users/' + nickname,
'actor': localActorUrl(httpPrefix, nickname, domain),
'to': [toUrl],
'cc': [],
'object': objectJson
@ -160,10 +162,16 @@ def _acceptFollow(baseDir: str, domain: str, messageJson: {},
' but they have been unfollowed')
return
# does the url path indicate that this is a group actor
groupAccount = hasGroupType(baseDir, followedActor, None, debug)
if debug:
print('Accepted follow is a group: ' + str(groupAccount) +
' ' + followedActor + ' ' + baseDir)
if followPerson(baseDir,
nickname, acceptedDomainFull,
followedNickname, followedDomainFull,
federationList, debug):
federationList, debug, groupAccount):
if debug:
print('DEBUG: ' + nickname + '@' + acceptedDomainFull +
' followed ' + followedNickname + '@' + followedDomainFull)

View File

@ -7,6 +7,7 @@ __email__ = "bob@freedombone.net"
__status__ = "Production"
__module_group__ = "ActivityPub"
from utils import hasGroupType
from utils import removeDomainPort
from utils import hasObjectDict
from utils import removeIdEnding
@ -21,6 +22,7 @@ from utils import locatePost
from utils import saveJson
from utils import undoAnnounceCollectionEntry
from utils import updateAnnounceCollection
from utils import localActorUrl
from posts import sendSignedJson
from posts import getPersonBox
from session import postJson
@ -135,11 +137,11 @@ def createAnnounce(session, baseDir: str, federationList: [],
statusNumber, published = getStatusNumber()
newAnnounceId = httpPrefix + '://' + fullDomain + \
'/users/' + nickname + '/statuses/' + statusNumber
atomUriStr = httpPrefix + '://' + fullDomain + '/users/' + nickname + \
atomUriStr = localActorUrl(httpPrefix, nickname, fullDomain) + \
'/statuses/' + statusNumber
newAnnounce = {
"@context": "https://www.w3.org/ns/activitystreams",
'actor': httpPrefix + '://' + fullDomain + '/users/' + nickname,
'actor': localActorUrl(httpPrefix, nickname, fullDomain),
'atomUri': atomUriStr,
'cc': [],
'id': newAnnounceId + '/activity',
@ -159,9 +161,16 @@ def createAnnounce(session, baseDir: str, federationList: [],
announceNickname = None
announceDomain = None
announcePort = None
groupAccount = False
if hasUsersPath(objectUrl):
announceNickname = getNicknameFromActor(objectUrl)
announceDomain, announcePort = getDomainFromActor(objectUrl)
if '/' + str(announceNickname) + '/' in objectUrl:
announceActor = \
objectUrl.split('/' + announceNickname + '/')[0] + \
'/' + announceNickname
if hasGroupType(baseDir, announceActor, personCache):
groupAccount = True
if announceNickname and announceDomain:
sendSignedJson(newAnnounce, session, baseDir,
@ -169,7 +178,7 @@ def createAnnounce(session, baseDir: str, federationList: [],
announceNickname, announceDomain, announcePort, None,
httpPrefix, True, clientToServer, federationList,
sendThreads, postLog, cachedWebfingers, personCache,
debug, projectVersion)
debug, projectVersion, None, groupAccount)
return newAnnounce
@ -185,8 +194,7 @@ def announcePublic(session, baseDir: str, federationList: [],
fromDomain = getFullDomain(domain, port)
toUrl = 'https://www.w3.org/ns/activitystreams#Public'
ccUrl = httpPrefix + '://' + fromDomain + '/users/' + nickname + \
'/followers'
ccUrl = localActorUrl(httpPrefix, nickname, fromDomain) + '/followers'
return createAnnounce(session, baseDir, federationList,
nickname, domain, port,
toUrl, ccUrl, httpPrefix,
@ -211,13 +219,11 @@ def sendAnnounceViaServer(baseDir: str, session,
fromDomainFull = getFullDomain(fromDomain, fromPort)
toUrl = 'https://www.w3.org/ns/activitystreams#Public'
ccUrl = httpPrefix + '://' + fromDomainFull + '/users/' + fromNickname + \
'/followers'
actorStr = localActorUrl(httpPrefix, fromNickname, fromDomainFull)
ccUrl = actorStr + '/followers'
statusNumber, published = getStatusNumber()
newAnnounceId = httpPrefix + '://' + fromDomainFull + '/users/' + \
fromNickname + '/statuses/' + statusNumber
actorStr = httpPrefix + '://' + fromDomainFull + '/users/' + fromNickname
newAnnounceId = actorStr + '/statuses/' + statusNumber
newAnnounceJson = {
"@context": "https://www.w3.org/ns/activitystreams",
'actor': actorStr,
@ -235,7 +241,7 @@ def sendAnnounceViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
fromDomain, projectVersion, debug)
fromDomain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: announce webfinger failed for ' + handle)
@ -300,7 +306,7 @@ def sendUndoAnnounceViaServer(baseDir: str, session,
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
handle = actor.replace('/users/', '/@')
statusNumber, published = getStatusNumber()
@ -315,7 +321,7 @@ def sendUndoAnnounceViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
domain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: undo announce webfinger failed for ' + handle)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 53 KiB

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 163 KiB

After

Width:  |  Height:  |  Size: 182 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 54 KiB

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 62 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 133 KiB

After

Width:  |  Height:  |  Size: 172 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 87 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 162 KiB

After

Width:  |  Height:  |  Size: 184 KiB

View File

@ -89,7 +89,7 @@ def authorizeBasic(baseDir: str, path: str, authHeader: str,
"""
if ' ' not in authHeader:
if debug:
print('DEBUG: basic auth - Authorixation header does not ' +
print('DEBUG: basic auth - Authorisation header does not ' +
'contain a space character')
return False
if not hasUsersPath(path):
@ -132,9 +132,10 @@ def authorizeBasic(baseDir: str, path: str, authHeader: str,
print('DEBUG: passwords file missing')
return False
providedPassword = plain.split(':')[1]
passfile = open(passwordFile, 'r')
for line in passfile:
if line.startswith(nickname + ':'):
with open(passwordFile, 'r') as passfile:
for line in passfile:
if not line.startswith(nickname + ':'):
continue
storedPassword = \
line.split(':')[1].replace('\n', '').replace('\r', '')
success = _verifyPassword(storedPassword, providedPassword)

View File

@ -18,6 +18,7 @@ from utils import getDomainFromActor
from utils import loadJson
from utils import saveJson
from utils import acctDir
from utils import localActorUrl
def setAvailability(baseDir: str, nickname: str, domain: str,
@ -90,13 +91,12 @@ def sendAvailabilityViaServer(baseDir: str, session,
domainFull = getFullDomain(domain, port)
toUrl = httpPrefix + '://' + domainFull + '/users/' + nickname
ccUrl = httpPrefix + '://' + domainFull + '/users/' + nickname + \
'/followers'
toUrl = localActorUrl(httpPrefix, nickname, domainFull)
ccUrl = toUrl + '/followers'
newAvailabilityJson = {
'type': 'Availability',
'actor': httpPrefix + '://' + domainFull + '/users/' + nickname,
'actor': toUrl,
'object': '"' + status + '"',
'to': [toUrl],
'cc': [ccUrl]
@ -107,7 +107,7 @@ def sendAvailabilityViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
domain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: availability webfinger failed for ' + handle)

View File

@ -28,6 +28,9 @@ from utils import evilIncarnate
from utils import getDomainFromActor
from utils import getNicknameFromActor
from utils import acctDir
from utils import localActorUrl
from conversation import muteConversation
from conversation import unmuteConversation
def addGlobalBlock(baseDir: str,
@ -60,12 +63,39 @@ def addBlock(baseDir: str, nickname: str, domain: str,
blockNickname: str, blockDomain: str) -> bool:
"""Block the given account
"""
if blockDomain.startswith(domain) and nickname == blockNickname:
# don't block self
return False
domain = removeDomainPort(domain)
blockingFilename = acctDir(baseDir, nickname, domain) + '/blocking.txt'
blockHandle = blockNickname + '@' + blockDomain
if os.path.isfile(blockingFilename):
if blockHandle in open(blockingFilename).read():
if blockHandle + '\n' in open(blockingFilename).read():
return False
# if we are following then unfollow
followingFilename = acctDir(baseDir, nickname, domain) + '/following.txt'
if os.path.isfile(followingFilename):
if blockHandle + '\n' in open(followingFilename).read():
followingStr = ''
with open(followingFilename, 'r') as followingFile:
followingStr = followingFile.read()
followingStr = followingStr.replace(blockHandle + '\n', '')
with open(followingFilename, 'w+') as followingFile:
followingFile.write(followingStr)
# if they are a follower then remove them
followersFilename = acctDir(baseDir, nickname, domain) + '/followers.txt'
if os.path.isfile(followersFilename):
if blockHandle + '\n' in open(followersFilename).read():
followersStr = ''
with open(followersFilename, 'r') as followersFile:
followersStr = followersFile.read()
followersStr = followersStr.replace(blockHandle + '\n', '')
with open(followersFilename, 'w+') as followersFile:
followersFile.write(followersStr)
with open(blockingFilename, 'a+') as blockFile:
blockFile.write(blockHandle + '\n')
return True
@ -305,25 +335,25 @@ def isBlocked(baseDir: str, nickname: str, domain: str,
def outboxBlock(baseDir: str, httpPrefix: str,
nickname: str, domain: str, port: int,
messageJson: {}, debug: bool) -> None:
messageJson: {}, debug: bool) -> bool:
""" When a block request is received by the outbox from c2s
"""
if not messageJson.get('type'):
if debug:
print('DEBUG: block - no type')
return
return False
if not messageJson['type'] == 'Block':
if debug:
print('DEBUG: not a block')
return
return False
if not messageJson.get('object'):
if debug:
print('DEBUG: no object in block')
return
return False
if not isinstance(messageJson['object'], str):
if debug:
print('DEBUG: block object is not string')
return
return False
if debug:
print('DEBUG: c2s block request arrived in outbox')
@ -331,22 +361,22 @@ def outboxBlock(baseDir: str, httpPrefix: str,
if '/statuses/' not in messageId:
if debug:
print('DEBUG: c2s block object is not a status')
return
return False
if not hasUsersPath(messageId):
if debug:
print('DEBUG: c2s block object has no nickname')
return
return False
domain = removeDomainPort(domain)
postFilename = locatePost(baseDir, nickname, domain, messageId)
if not postFilename:
if debug:
print('DEBUG: c2s block post not found in inbox or outbox')
print(messageId)
return
return False
nicknameBlocked = getNicknameFromActor(messageJson['object'])
if not nicknameBlocked:
print('WARN: unable to find nickname in ' + messageJson['object'])
return
return False
domainBlocked, portBlocked = getDomainFromActor(messageJson['object'])
domainBlockedFull = getFullDomain(domainBlocked, portBlocked)
@ -355,6 +385,7 @@ def outboxBlock(baseDir: str, httpPrefix: str,
if debug:
print('DEBUG: post blocked via c2s - ' + postFilename)
return True
def outboxUndoBlock(baseDir: str, httpPrefix: str,
@ -439,7 +470,12 @@ def mutePost(baseDir: str, nickname: str, domain: str, port: int,
if hasObjectDict(postJsonObject):
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
if postJsonObject['object'].get('conversation'):
muteConversation(baseDir, nickname, domain,
postJsonObject['object']['conversation'])
# does this post have ignores on it from differenent actors?
if not postJsonObject['object'].get('ignores'):
if debug:
@ -518,9 +554,13 @@ def unmutePost(baseDir: str, nickname: str, domain: str, port: int,
print('UNMUTE: ' + muteFilename + ' file removed')
if hasObjectDict(postJsonObject):
if postJsonObject['object'].get('conversation'):
unmuteConversation(baseDir, nickname, domain,
postJsonObject['object']['conversation'])
if postJsonObject['object'].get('ignores'):
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
totalItems = 0
if postJsonObject['object']['ignores'].get('totalItems'):
totalItems = \

89
blog.py
View File

@ -15,7 +15,12 @@ from webapp_utils import htmlHeaderWithExternalStyle
from webapp_utils import htmlHeaderWithBlogMarkup
from webapp_utils import htmlFooter
from webapp_utils import getPostAttachmentsAsHtml
from webapp_utils import editTextArea
from webapp_media import addEmbeddedElements
from utils import localActorUrl
from utils import getActorLanguagesList
from utils import getBaseContentFromPost
from utils import getContentFromPost
from utils import isAccountDir
from utils import removeHtml
from utils import getConfigParam
@ -31,6 +36,7 @@ from utils import acctDir
from posts import createBlogsTimeline
from newswire import rss2Header
from newswire import rss2Footer
from cache import getPersonFromCache
def _noOfBlogReplies(baseDir: str, httpPrefix: str, translate: {},
@ -164,7 +170,9 @@ def _htmlBlogPostContent(authorized: bool,
postJsonObject: {},
handle: str, restrictToDomain: bool,
peertubeInstances: [],
blogSeparator='<hr>') -> str:
systemLanguage: str,
personCache: {},
blogSeparator: str = '<hr>') -> str:
"""Returns the content for a single blog post
"""
linkedAuthor = False
@ -235,9 +243,16 @@ def _htmlBlogPostContent(authorized: bool,
if attachmentStr:
blogStr += '<br><center>' + attachmentStr + '</center>'
if postJsonObject['object'].get('content'):
contentStr = addEmbeddedElements(translate,
postJsonObject['object']['content'],
personUrl = localActorUrl(httpPrefix, nickname, domainFull)
actorJson = \
getPersonFromCache(baseDir, personUrl, personCache, False)
languagesUnderstood = []
if actorJson:
languagesUnderstood = getActorLanguagesList(actorJson)
jsonContent = getContentFromPost(postJsonObject, systemLanguage,
languagesUnderstood)
if jsonContent:
contentStr = addEmbeddedElements(translate, jsonContent,
peertubeInstances)
if postJsonObject['object'].get('tag'):
contentStr = replaceEmojiFromTags(contentStr,
@ -273,8 +288,8 @@ def _htmlBlogPostContent(authorized: bool,
if not linkedAuthor:
blogStr += '<p class="about"><a class="about" href="' + \
httpPrefix + '://' + domainFull + \
'/users/' + nickname + '">' + translate['About the author'] + \
localActorUrl(httpPrefix, nickname, domainFull) + \
'">' + translate['About the author'] + \
'</a></p>\n'
replies = _noOfBlogReplies(baseDir, httpPrefix, translate,
@ -312,7 +327,8 @@ def _htmlBlogPostRSS2(authorized: bool,
baseDir: str, httpPrefix: str, translate: {},
nickname: str, domain: str, domainFull: str,
postJsonObject: {},
handle: str, restrictToDomain: bool) -> str:
handle: str, restrictToDomain: bool,
systemLanguage: str) -> str:
"""Returns the RSS version 2 feed for a single blog post
"""
rssStr = ''
@ -327,7 +343,8 @@ def _htmlBlogPostRSS2(authorized: bool,
pubDate = datetime.strptime(published, "%Y-%m-%dT%H:%M:%SZ")
titleStr = postJsonObject['object']['summary']
rssDateStr = pubDate.strftime("%a, %d %b %Y %H:%M:%S UT")
content = postJsonObject['object']['content']
content = \
getBaseContentFromPost(postJsonObject, systemLanguage)
description = firstParagraphFromString(content)
rssStr = ' <item>'
rssStr += ' <title>' + titleStr + '</title>'
@ -343,7 +360,8 @@ def _htmlBlogPostRSS3(authorized: bool,
baseDir: str, httpPrefix: str, translate: {},
nickname: str, domain: str, domainFull: str,
postJsonObject: {},
handle: str, restrictToDomain: bool) -> str:
handle: str, restrictToDomain: bool,
systemLanguage: str) -> str:
"""Returns the RSS version 3 feed for a single blog post
"""
rssStr = ''
@ -358,7 +376,8 @@ def _htmlBlogPostRSS3(authorized: bool,
pubDate = datetime.strptime(published, "%Y-%m-%dT%H:%M:%SZ")
titleStr = postJsonObject['object']['summary']
rssDateStr = pubDate.strftime("%a, %d %b %Y %H:%M:%S UT")
content = postJsonObject['object']['content']
content = \
getBaseContentFromPost(postJsonObject, systemLanguage)
description = firstParagraphFromString(content)
rssStr = 'title: ' + titleStr + '\n'
rssStr += 'link: ' + messageLink + '\n'
@ -379,10 +398,10 @@ def _htmlBlogRemoveCwButton(blogStr: str, translate: {}) -> str:
return blogStr
def _getSnippetFromBlogContent(postJsonObject: {}) -> str:
def _getSnippetFromBlogContent(postJsonObject: {}, systemLanguage: str) -> str:
"""Returns a snippet of text from the blog post as a preview
"""
content = postJsonObject['object']['content']
content = getBaseContentFromPost(postJsonObject, systemLanguage)
if '<p>' in content:
content = content.split('<p>', 1)[1]
if '</p>' in content:
@ -400,7 +419,7 @@ def htmlBlogPost(authorized: bool,
nickname: str, domain: str, domainFull: str,
postJsonObject: {},
peertubeInstances: [],
systemLanguage: str) -> str:
systemLanguage: str, personCache: {}) -> str:
"""Returns a html blog post
"""
blogStr = ''
@ -412,7 +431,7 @@ def htmlBlogPost(authorized: bool,
getConfigParam(baseDir, 'instanceTitle')
published = postJsonObject['object']['published']
title = postJsonObject['object']['summary']
snippet = _getSnippetFromBlogContent(postJsonObject)
snippet = _getSnippetFromBlogContent(postJsonObject, systemLanguage)
blogStr = htmlHeaderWithBlogMarkup(cssFilename, instanceTitle,
httpPrefix, domainFull, nickname,
systemLanguage, published,
@ -424,7 +443,8 @@ def htmlBlogPost(authorized: bool,
nickname, domain,
domainFull, postJsonObject,
None, False,
peertubeInstances)
peertubeInstances, systemLanguage,
personCache)
# show rss links
blogStr += '<p class="rssfeed">'
@ -452,7 +472,8 @@ def htmlBlogPage(authorized: bool, session,
baseDir: str, httpPrefix: str, translate: {},
nickname: str, domain: str, port: int,
noOfItems: int, pageNumber: int,
peertubeInstances: []) -> str:
peertubeInstances: [], systemLanguage: str,
personCache: {}) -> str:
"""Returns a html blog page containing posts
"""
if ' ' in nickname or '@' in nickname or \
@ -514,7 +535,9 @@ def htmlBlogPage(authorized: bool, session,
nickname, domain,
domainFull, item,
None, True,
peertubeInstances)
peertubeInstances,
systemLanguage,
personCache)
if len(timelineJson['orderedItems']) >= noOfItems:
blogStr += navigateStr
@ -542,7 +565,7 @@ def htmlBlogPageRSS2(authorized: bool, session,
baseDir: str, httpPrefix: str, translate: {},
nickname: str, domain: str, port: int,
noOfItems: int, pageNumber: int,
includeHeader: bool) -> str:
includeHeader: bool, systemLanguage: str) -> str:
"""Returns an RSS version 2 feed containing posts
"""
if ' ' in nickname or '@' in nickname or \
@ -585,7 +608,7 @@ def htmlBlogPageRSS2(authorized: bool, session,
httpPrefix, translate,
nickname, domain,
domainFull, item,
None, True)
None, True, systemLanguage)
if includeHeader:
return blogRSS2 + rss2Footer()
@ -596,7 +619,8 @@ def htmlBlogPageRSS2(authorized: bool, session,
def htmlBlogPageRSS3(authorized: bool, session,
baseDir: str, httpPrefix: str, translate: {},
nickname: str, domain: str, port: int,
noOfItems: int, pageNumber: int) -> str:
noOfItems: int, pageNumber: int,
systemLanguage: str) -> str:
"""Returns an RSS version 3 feed containing posts
"""
if ' ' in nickname or '@' in nickname or \
@ -630,7 +654,8 @@ def htmlBlogPageRSS3(authorized: bool, session,
httpPrefix, translate,
nickname, domain,
domainFull, item,
None, True)
None, True,
systemLanguage)
return blogRSS3
@ -670,7 +695,8 @@ def htmlBlogView(authorized: bool,
session, baseDir: str, httpPrefix: str,
translate: {}, domain: str, port: int,
noOfItems: int,
peertubeInstances: []) -> str:
peertubeInstances: [], systemLanguage: str,
personCache: {}) -> str:
"""Show the blog main page
"""
blogStr = ''
@ -688,7 +714,8 @@ def htmlBlogView(authorized: bool,
return htmlBlogPage(authorized, session,
baseDir, httpPrefix, translate,
nickname, domain, port,
noOfItems, 1, peertubeInstances)
noOfItems, 1, peertubeInstances,
systemLanguage, personCache)
domainFull = getFullDomain(domain, port)
@ -714,7 +741,7 @@ def htmlEditBlog(mediaInstance: bool, translate: {},
path: str,
pageNumber: int,
nickname: str, domain: str,
postUrl: str) -> str:
postUrl: str, systemLanguage: str) -> str:
"""Edit a blog post after it was created
"""
postFilename = locatePost(baseDir, nickname, domain, postUrl)
@ -828,17 +855,15 @@ def htmlEditBlog(mediaInstance: bool, translate: {},
editBlogForm += \
' <input type="text" name="subject" value="' + titleStr + '">'
editBlogForm += ''
editBlogForm += ' <br><label class="labels">' + \
placeholderMessage + '</label>'
editBlogForm += ' <br>'
messageBoxHeight = 800
contentStr = postJsonObject['object']['content']
contentStr = getBaseContentFromPost(postJsonObject, systemLanguage)
contentStr = contentStr.replace('<p>', '').replace('</p>', '\n')
editBlogForm += \
' <textarea id="message" name="message" style="height:' + \
str(messageBoxHeight) + 'px" spellcheck="true">' + \
contentStr + '</textarea>'
editTextArea(placeholderMessage, 'message', contentStr,
messageBoxHeight, '', True)
editBlogForm += dateAndLocation
if not mediaInstance:
editBlogForm += editBlogImageSection
@ -877,8 +902,8 @@ def pathContainsBlogLink(baseDir: str,
return None, None
if '#' + userEnding2[1] + '.' not in open(blogIndexFilename).read():
return None, None
messageId = httpPrefix + '://' + domainFull + \
'/users/' + nickname + '/statuses/' + userEnding2[1]
messageId = localActorUrl(httpPrefix, nickname, domainFull) + \
'/statuses/' + userEnding2[1]
return locatePost(baseDir, nickname, domain, messageId), nickname

View File

@ -25,6 +25,7 @@ from utils import loadJson
from utils import saveJson
from utils import hasObjectDict
from utils import acctDir
from utils import localActorUrl
from posts import getPersonBox
from session import postJson
@ -242,7 +243,7 @@ def bookmark(recentPostsCache: {},
newBookmarkJson = {
"@context": "https://www.w3.org/ns/activitystreams",
'type': 'Bookmark',
'actor': httpPrefix + '://' + fullDomain + '/users/' + nickname,
'actor': localActorUrl(httpPrefix, nickname, fullDomain),
'object': objectUrl
}
if ccList:
@ -301,10 +302,10 @@ def undoBookmark(recentPostsCache: {},
newUndoBookmarkJson = {
"@context": "https://www.w3.org/ns/activitystreams",
'type': 'Undo',
'actor': httpPrefix + '://' + fullDomain + '/users/' + nickname,
'actor': localActorUrl(httpPrefix, nickname, fullDomain),
'object': {
'type': 'Bookmark',
'actor': httpPrefix + '://' + fullDomain + '/users/' + nickname,
'actor': localActorUrl(httpPrefix, nickname, fullDomain),
'object': objectUrl
}
}
@ -356,7 +357,7 @@ def sendBookmarkViaServer(baseDir: str, session,
domainFull = getFullDomain(domain, fromPort)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
newBookmarkJson = {
"@context": "https://www.w3.org/ns/activitystreams",
@ -376,7 +377,7 @@ def sendBookmarkViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
domain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: bookmark webfinger failed for ' + handle)
@ -441,7 +442,7 @@ def sendUndoBookmarkViaServer(baseDir: str, session,
domainFull = getFullDomain(domain, fromPort)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
newBookmarkJson = {
"@context": "https://www.w3.org/ns/activitystreams",
@ -461,7 +462,7 @@ def sendUndoBookmarkViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
domain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: unbookmark webfinger failed for ' + handle)

View File

@ -10,9 +10,11 @@ __module_group__ = "Core"
import os
import datetime
from session import urlExists
from session import getJson
from utils import loadJson
from utils import saveJson
from utils import getFileCaseInsensitive
from utils import getUserPaths
def _removePersonFromCache(baseDir: str, personUrl: str,
@ -132,3 +134,52 @@ def getWebfingerFromCache(handle: str, cachedWebfingers: {}) -> {}:
if cachedWebfingers.get(handle):
return cachedWebfingers[handle]
return None
def getPersonPubKey(baseDir: str, session, personUrl: str,
personCache: {}, debug: bool,
projectVersion: str, httpPrefix: str,
domain: str, onionDomain: str) -> str:
if not personUrl:
return None
personUrl = personUrl.replace('#main-key', '')
usersPaths = getUserPaths()
for possibleUsersPath in usersPaths:
if personUrl.endswith(possibleUsersPath + 'inbox'):
if debug:
print('DEBUG: Obtaining public key for shared inbox')
personUrl = \
personUrl.replace(possibleUsersPath + 'inbox', '/inbox')
break
personJson = \
getPersonFromCache(baseDir, personUrl, personCache, True)
if not personJson:
if debug:
print('DEBUG: Obtaining public key for ' + personUrl)
personDomain = domain
if onionDomain:
if '.onion/' in personUrl:
personDomain = onionDomain
profileStr = 'https://www.w3.org/ns/activitystreams'
asHeader = {
'Accept': 'application/activity+json; profile="' + profileStr + '"'
}
personJson = \
getJson(session, personUrl, asHeader, None, debug,
projectVersion, httpPrefix, personDomain)
if not personJson:
return None
pubKey = None
if personJson.get('publicKey'):
if personJson['publicKey'].get('publicKeyPem'):
pubKey = personJson['publicKey']['publicKeyPem']
else:
if personJson.get('publicKeyPem'):
pubKey = personJson['publicKeyPem']
if not pubKey:
if debug:
print('DEBUG: Public key not found for ' + personUrl)
storePersonInCache(baseDir, personUrl, personJson, personCache, True)
return pubKey

View File

@ -86,7 +86,7 @@ def getHashtagCategories(baseDir: str,
return hashtagCategories
def _updateHashtagCategories(baseDir: str) -> None:
def updateHashtagCategories(baseDir: str) -> None:
"""Regenerates the list of hashtag categories
"""
categoryListFilename = baseDir + '/accounts/categoryList.txt'
@ -129,7 +129,7 @@ def _validHashtagCategory(category: str) -> bool:
def setHashtagCategory(baseDir: str, hashtag: str, category: str,
force: bool = False) -> bool:
update: bool, force: bool = False) -> bool:
"""Sets the category for the hashtag
"""
if not _validHashtagCategory(category):
@ -155,7 +155,8 @@ def setHashtagCategory(baseDir: str, hashtag: str, category: str,
return False
with open(categoryFilename, 'w+') as fp:
fp.write(category)
_updateHashtagCategories(baseDir)
if update:
updateHashtagCategories(baseDir)
return True
return False

View File

@ -292,6 +292,7 @@ def getSpoofedCity(city: str, baseDir: str, nickname: str, domain: str) -> str:
"""Returns the name of the city to use as a GPS spoofing location for
image metadata
"""
city = ''
cityFilename = acctDir(baseDir, nickname, domain) + '/city.txt'
if os.path.isfile(cityFilename):
with open(cityFilename, 'r') as fp:

View File

@ -21,6 +21,8 @@ from utils import dangerousMarkup
from utils import isPGPEncrypted
from utils import containsPGPPublicKey
from utils import acctDir
from utils import isfloat
from utils import getCurrencies
from petnames import getPetName
@ -497,7 +499,7 @@ def _addMention(wordStr: str, httpPrefix: str, following: str, petnames: str,
followStr = follow.replace('\n', '').replace('\r', '')
replaceDomain = followStr.split('@')[1]
recipientActor = httpPrefix + "://" + \
replaceDomain + "/users/" + possibleNickname
replaceDomain + "/@" + possibleNickname
if recipientActor not in recipients:
recipients.append(recipientActor)
tags[wordStr] = {
@ -524,7 +526,7 @@ def _addMention(wordStr: str, httpPrefix: str, following: str, petnames: str,
replaceNickname = followStr.split('@')[0]
replaceDomain = followStr.split('@')[1]
recipientActor = httpPrefix + "://" + \
replaceDomain + "/users/" + replaceNickname
replaceDomain + "/@" + replaceNickname
if recipientActor not in recipients:
recipients.append(recipientActor)
tags[wordStr] = {
@ -556,7 +558,7 @@ def _addMention(wordStr: str, httpPrefix: str, following: str, petnames: str,
if follow.replace('\n', '').replace('\r', '') != possibleHandle:
continue
recipientActor = httpPrefix + "://" + \
possibleDomain + "/users/" + possibleNickname
possibleDomain + "/@" + possibleNickname
if recipientActor not in recipients:
recipients.append(recipientActor)
tags[wordStr] = {
@ -574,7 +576,7 @@ def _addMention(wordStr: str, httpPrefix: str, following: str, petnames: str,
if not (possibleDomain == 'localhost' or '.' in possibleDomain):
return False
recipientActor = httpPrefix + "://" + \
possibleDomain + "/users/" + possibleNickname
possibleDomain + "/@" + possibleNickname
if recipientActor not in recipients:
recipients.append(recipientActor)
tags[wordStr] = {
@ -930,6 +932,16 @@ def saveMediaInFormPOST(mediaBytes, debug: bool,
Returns the filename and attachment type
"""
if not mediaBytes:
if filenameBase:
# remove any existing files
extensionTypes = getImageExtensions()
for ex in extensionTypes:
possibleOtherFormat = filenameBase + '.' + ex
if os.path.isfile(possibleOtherFormat):
os.remove(possibleOtherFormat)
if os.path.isfile(filenameBase):
os.remove(filenameBase)
if debug:
print('DEBUG: No media found within POST')
return None, None
@ -951,6 +963,7 @@ def saveMediaInFormPOST(mediaBytes, debug: bool,
'ogv': 'video/ogv',
'mp3': 'audio/mpeg',
'ogg': 'audio/ogg',
'flac': 'audio/flac',
'zip': 'application/zip'
}
detectedExtension = None
@ -1085,3 +1098,21 @@ def limitRepeatedWords(text: str, maxRepeats: int) -> str:
for word, item in replacements.items():
text = text.replace(item[0], item[1])
return text
def getPriceFromString(priceStr: str) -> (str, str):
"""Returns the item price and currency
"""
currencies = getCurrencies()
for symbol, name in currencies.items():
if symbol in priceStr:
price = priceStr.replace(symbol, '')
if isfloat(price):
return price, name
elif name in priceStr:
price = priceStr.replace(name, '')
if isfloat(price):
return price, name
if isfloat(priceStr):
return priceStr, "EUR"
return "0.00", "EUR"

View File

@ -14,6 +14,7 @@ validContexts = (
"https://w3id.org/security/v1",
"*/apschema/v1.9",
"*/apschema/v1.21",
"*/apschema/v1.20",
"*/litepub-0.1.jsonld",
"https://litepub.social/litepub/context.jsonld"
)
@ -100,6 +101,43 @@ def getApschemaV1_9() -> {}:
}
def getApschemaV1_20() -> {}:
# https://domain/apschema/v1.20
return {
"@context":
{
"as": "https://www.w3.org/ns/activitystreams#",
"zot": "https://zap.dog/apschema#",
"toot": "http://joinmastodon.org/ns#",
"ostatus": "http://ostatus.org#",
"schema": "http://schema.org#",
"litepub": "http://litepub.social/ns#",
"sm": "http://smithereen.software/ns#",
"conversation": "ostatus:conversation",
"manuallyApprovesFollowers": "as:manuallyApprovesFollowers",
"oauthRegistrationEndpoint": "litepub:oauthRegistrationEndpoint",
"sensitive": "as:sensitive",
"movedTo": "as:movedTo",
"copiedTo": "as:copiedTo",
"alsoKnownAs": "as:alsoKnownAs",
"inheritPrivacy": "as:inheritPrivacy",
"EmojiReact": "as:EmojiReact",
"commentPolicy": "zot:commentPolicy",
"topicalCollection": "zot:topicalCollection",
"eventRepeat": "zot:eventRepeat",
"emojiReaction": "zot:emojiReaction",
"expires": "zot:expires",
"directMessage": "zot:directMessage",
"Category": "zot:Category",
"replyTo": "zot:replyTo",
"PropertyValue": "schema:PropertyValue",
"value": "schema:value",
"discoverable": "toot:discoverable",
"wall": "sm:wall"
}
}
def getApschemaV1_21() -> {}:
# https://domain/apschema/v1.21
return {

78
conversation.py 100644
View File

@ -0,0 +1,78 @@
__filename__ = "conversation.py"
__author__ = "Bob Mottram"
__license__ = "AGPL3+"
__version__ = "1.2.0"
__maintainer__ = "Bob Mottram"
__email__ = "bob@freedombone.net"
__status__ = "Production"
__module_group__ = "Timeline"
import os
from utils import hasObjectDict
from utils import acctDir
def updateConversation(baseDir: str, nickname: str, domain: str,
postJsonObject: {}) -> bool:
"""Ads a post to a conversation index in the /conversation subdirectory
"""
if not hasObjectDict(postJsonObject):
return False
if not postJsonObject['object'].get('conversation'):
return False
if not postJsonObject['object'].get('id'):
return False
conversationDir = acctDir(baseDir, nickname, domain) + '/conversation'
if not os.path.isdir(conversationDir):
os.mkdir(conversationDir)
conversationId = postJsonObject['object']['conversation']
conversationId = conversationId.replace('/', '#')
postId = postJsonObject['object']['id']
conversationFilename = conversationDir + '/' + conversationId
if not os.path.isfile(conversationFilename):
try:
with open(conversationFilename, 'w+') as fp:
fp.write(postId + '\n')
return True
except BaseException:
pass
elif postId + '\n' not in open(conversationFilename).read():
try:
with open(conversationFilename, 'a+') as fp:
fp.write(postId + '\n')
return True
except BaseException:
pass
return False
def muteConversation(baseDir: str, nickname: str, domain: str,
conversationId: str) -> None:
"""Mutes the given conversation
"""
conversationDir = acctDir(baseDir, nickname, domain) + '/conversation'
conversationFilename = \
conversationDir + '/' + conversationId.replace('/', '#')
if not os.path.isfile(conversationFilename):
return
if os.path.isfile(conversationFilename + '.muted'):
return
with open(conversationFilename + '.muted', 'w+') as fp:
fp.write('\n')
def unmuteConversation(baseDir: str, nickname: str, domain: str,
conversationId: str) -> None:
"""Unmutes the given conversation
"""
conversationDir = acctDir(baseDir, nickname, domain) + '/conversation'
conversationFilename = \
conversationDir + '/' + conversationId.replace('/', '#')
if not os.path.isfile(conversationFilename):
return
if not os.path.isfile(conversationFilename + '.muted'):
return
try:
os.remove(conversationFilename + '.muted')
except BaseException:
pass

1795
daemon.py

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@ -18,6 +18,7 @@ from utils import getDomainFromActor
from utils import locatePost
from utils import deletePost
from utils import removeModerationPostFromIndex
from utils import localActorUrl
from session import postJson
from webfinger import webfingerHandle
from auth import createBasicAuthHeader
@ -38,8 +39,7 @@ def sendDeleteViaServer(baseDir: str, session,
fromDomainFull = getFullDomain(fromDomain, fromPort)
actor = httpPrefix + '://' + fromDomainFull + \
'/users/' + fromNickname
actor = localActorUrl(httpPrefix, fromNickname, fromDomainFull)
toUrl = 'https://www.w3.org/ns/activitystreams#Public'
ccUrl = actor + '/followers'
@ -57,7 +57,7 @@ def sendDeleteViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = \
webfingerHandle(session, handle, httpPrefix, cachedWebfingers,
fromDomain, projectVersion, debug)
fromDomain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: delete webfinger failed for ' + handle)

View File

@ -16,6 +16,7 @@ import webbrowser
import urllib.parse
from pathlib import Path
from random import randint
from utils import getBaseContentFromPost
from utils import hasObjectDict
from utils import getFullDomain
from utils import isDM
@ -24,6 +25,7 @@ from utils import removeHtml
from utils import getNicknameFromActor
from utils import getDomainFromActor
from utils import isPGPEncrypted
from utils import localActorUrl
from session import createSession
from speaker import speakableText
from speaker import getSpeakerPitch
@ -415,7 +417,8 @@ def _desktopReplyToPost(session, postId: str,
cachedWebfingers: {}, personCache: {},
debug: bool, subject: str,
screenreader: str, systemLanguage: str,
espeak) -> None:
espeak, conversationId: str,
lowBandwidth: bool) -> None:
"""Use the desktop client to send a reply to the most recent post
"""
if '://' not in postId:
@ -468,7 +471,9 @@ def _desktopReplyToPost(session, postId: str,
commentsEnabled, attach, mediaType,
attachedImageDescription, city,
cachedWebfingers, personCache, isArticle,
debug, postId, postId, subject) == 0:
systemLanguage, lowBandwidth,
debug, postId, postId,
conversationId, subject) == 0:
sayStr = 'Reply sent'
else:
sayStr = 'Reply failed'
@ -481,9 +486,10 @@ def _desktopNewPost(session,
cachedWebfingers: {}, personCache: {},
debug: bool,
screenreader: str, systemLanguage: str,
espeak) -> None:
espeak, lowBandwidth: bool) -> None:
"""Use the desktop client to create a new post
"""
conversationId = None
sayStr = 'Create new post'
_sayCommand(sayStr, sayStr, screenreader, systemLanguage, espeak)
sayStr = 'Type your post, then press Enter.'
@ -529,7 +535,9 @@ def _desktopNewPost(session,
commentsEnabled, attach, mediaType,
attachedImageDescription, city,
cachedWebfingers, personCache, isArticle,
debug, None, None, subject) == 0:
systemLanguage, lowBandwidth,
debug, None, None,
conversationId, subject) == 0:
sayStr = 'Post sent'
else:
sayStr = 'Post failed'
@ -652,7 +660,8 @@ def _readLocalBoxPost(session, nickname: str, domain: str,
pageNumber: int, index: int, boxJson: {},
systemLanguage: str,
screenreader: str, espeak,
translate: {}, yourActor: str) -> {}:
translate: {}, yourActor: str,
domainFull: str, personCache: {}) -> {}:
"""Reads a post from the given timeline
Returns the post json
"""
@ -687,15 +696,17 @@ def _readLocalBoxPost(session, nickname: str, domain: str,
__version__, translate,
YTReplacementDomain,
allowLocalNetworkAccess,
recentPostsCache, False)
recentPostsCache, False,
systemLanguage,
domainFull, personCache)
if postJsonObject2:
if hasObjectDict(postJsonObject2):
if postJsonObject2['object'].get('attributedTo') and \
postJsonObject2['object'].get('content'):
attributedTo = postJsonObject2['object']['attributedTo']
content = postJsonObject2['object']['content']
if isinstance(attributedTo, str) and \
isinstance(content, str):
content = \
getBaseContentFromPost(postJsonObject2, systemLanguage)
if isinstance(attributedTo, str) and content:
actor = attributedTo
nameStr += ' ' + translate['announces'] + ' ' + \
getNicknameFromActor(actor)
@ -719,7 +730,7 @@ def _readLocalBoxPost(session, nickname: str, domain: str,
attributedTo = postJsonObject['object']['attributedTo']
if not attributedTo:
return {}
content = postJsonObject['object']['content']
content = getBaseContentFromPost(postJsonObject, systemLanguage)
if not isinstance(attributedTo, str) or \
not isinstance(content, str):
return {}
@ -1042,7 +1053,8 @@ def _desktopShowBox(indent: str,
published = _formatPublished(postJsonObject['published'])
content = _textOnlyContent(postJsonObject['object']['content'])
contentStr = getBaseContentFromPost(postJsonObject, systemLanguage)
content = _textOnlyContent(contentStr)
if boxName != 'dm':
if isDM(postJsonObject):
content = '📧' + content
@ -1100,7 +1112,7 @@ def _desktopNewDM(session, toHandle: str,
cachedWebfingers: {}, personCache: {},
debug: bool,
screenreader: str, systemLanguage: str,
espeak) -> None:
espeak, lowBandwidth: bool) -> None:
"""Use the desktop client to create a new direct message
which can include multiple destination handles
"""
@ -1121,7 +1133,7 @@ def _desktopNewDM(session, toHandle: str,
cachedWebfingers, personCache,
debug,
screenreader, systemLanguage,
espeak)
espeak, lowBandwidth)
def _desktopNewDMbase(session, toHandle: str,
@ -1130,9 +1142,10 @@ def _desktopNewDMbase(session, toHandle: str,
cachedWebfingers: {}, personCache: {},
debug: bool,
screenreader: str, systemLanguage: str,
espeak) -> None:
espeak, lowBandwidth: bool) -> None:
"""Use the desktop client to create a new direct message
"""
conversationId = None
toPort = port
if '://' in toHandle:
toNickname = getNicknameFromActor(toHandle)
@ -1217,7 +1230,9 @@ def _desktopNewDMbase(session, toHandle: str,
commentsEnabled, attach, mediaType,
attachedImageDescription, city,
cachedWebfingers, personCache, isArticle,
debug, None, None, subject) == 0:
systemLanguage, lowBandwidth,
debug, None, None,
conversationId, subject) == 0:
sayStr = 'Direct message sent'
else:
sayStr = 'Direct message failed'
@ -1282,7 +1297,7 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
storeInboxPosts: bool,
showNewPosts: bool,
language: str,
debug: bool) -> None:
debug: bool, lowBandwidth: bool) -> None:
"""Runs the desktop and screen reader client,
which announces new inbox items
"""
@ -1360,7 +1375,7 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
systemLanguage, espeak)
domainFull = getFullDomain(domain, port)
yourActor = httpPrefix + '://' + domainFull + '/users/' + nickname
yourActor = localActorUrl(httpPrefix, nickname, domainFull)
actorJson = None
notifyJson = {
@ -1590,7 +1605,8 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
httpPrefix, baseDir, currTimeline,
pageNumber, postIndex, boxJson,
systemLanguage, screenreader,
espeak, translate, yourActor)
espeak, translate, yourActor,
domainFull, personCache)
print('')
sayStr = 'Press Enter to continue...'
sayStr2 = _highlightText(sayStr)
@ -1661,6 +1677,10 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
subject = None
if postJsonObject['object'].get('summary'):
subject = postJsonObject['object']['summary']
conversationId = None
if postJsonObject['object'].get('conversation'):
conversationId = \
postJsonObject['object']['conversation']
sessionReply = createSession(proxyType)
_desktopReplyToPost(sessionReply, postId,
baseDir, nickname, password,
@ -1668,7 +1688,8 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
cachedWebfingers, personCache,
debug, subject,
screenreader, systemLanguage,
espeak)
espeak, conversationId,
lowBandwidth)
refreshTimeline = True
print('')
elif (commandStr == 'post' or commandStr == 'p' or
@ -1702,7 +1723,7 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
cachedWebfingers, personCache,
debug,
screenreader, systemLanguage,
espeak)
espeak, lowBandwidth)
refreshTimeline = True
else:
# public post
@ -1712,7 +1733,7 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
cachedWebfingers, personCache,
debug,
screenreader, systemLanguage,
espeak)
espeak, lowBandwidth)
refreshTimeline = True
print('')
elif commandStr == 'like' or commandStr.startswith('like '):
@ -1929,8 +1950,8 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
blockDomain = blockHandle.split('@')[1]
blockNickname = blockHandle.split('@')[0]
blockActor = \
httpPrefix + '://' + blockDomain + \
'/users/' + blockNickname
localActorUrl(httpPrefix,
blockNickname, blockDomain)
if currIndex > 0 and boxJson and not blockActor:
postJsonObject = \
_desktopGetBoxPostObject(boxJson, currIndex)
@ -2318,11 +2339,14 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
__version__, translate,
YTReplacementDomain,
allowLocalNetworkAccess,
recentPostsCache, False)
recentPostsCache, False,
systemLanguage,
domainFull, personCache)
if postJsonObject2:
postJsonObject = postJsonObject2
if postJsonObject:
content = postJsonObject['object']['content']
content = \
getBaseContentFromPost(postJsonObject, systemLanguage)
messageStr, detectedLinks = \
speakableText(baseDir, content, translate)
linkOpened = False
@ -2378,7 +2402,9 @@ def runDesktopClient(baseDir: str, proxyType: str, httpPrefix: str,
print('')
if postJsonObject['object'].get('summary'):
print(postJsonObject['object']['summary'])
print(postJsonObject['object']['content'])
contentStr = getBaseContentFromPost(postJsonObject,
systemLanguage)
print(contentStr)
print('')
sayStr = 'Confirm delete, yes or no?'
_sayCommand(sayStr, sayStr, screenreader,

View File

@ -34,6 +34,7 @@ import os
from utils import loadJson
from utils import saveJson
from utils import acctDir
from utils import localActorUrl
def E2EEremoveDevice(baseDir: str, nickname: str, domain: str,
@ -142,7 +143,7 @@ def E2EEdevicesCollection(baseDir: str, nickname: str, domain: str,
personDir = acctDir(baseDir, nickname, domain)
if not os.path.isdir(personDir):
return {}
personId = httpPrefix + '://' + domainFull + '/users/' + nickname
personId = localActorUrl(httpPrefix, nickname, domainFull)
if not os.path.isdir(personDir + '/devices'):
os.mkdir(personDir + '/devices')
deviceList = []

View File

@ -8,12 +8,16 @@ __status__ = "Production"
__module_group__ = "Profile Metadata"
def _getDonationTypes() -> str:
def _getDonationTypes() -> []:
return ('patreon', 'paypal', 'gofundme', 'liberapay',
'kickstarter', 'indiegogo', 'crowdsupply',
'subscribestar')
def _getWebsiteStrings() -> []:
return ['www', 'website', 'web', 'homepage']
def getDonationUrl(actorJson: {}) -> str:
"""Returns a link used for donations
"""
@ -39,6 +43,28 @@ def getDonationUrl(actorJson: {}) -> str:
return ''
def getWebsite(actorJson: {}, translate: {}) -> str:
"""Returns a web address link
"""
if not actorJson.get('attachment'):
return ''
matchStrings = _getWebsiteStrings()
matchStrings.append(translate['Website'].lower())
for propertyValue in actorJson['attachment']:
if not propertyValue.get('name'):
continue
if propertyValue['name'].lower() not in matchStrings:
continue
if not propertyValue.get('type'):
continue
if not propertyValue.get('value'):
continue
if propertyValue['type'] != 'PropertyValue':
continue
return propertyValue['value']
return ''
def setDonationUrl(actorJson: {}, donateUrl: str) -> None:
"""Sets a link used for donations
"""
@ -102,3 +128,47 @@ def setDonationUrl(actorJson: {}, donateUrl: str) -> None:
"value": donateValue
}
actorJson['attachment'].append(newDonate)
def setWebsite(actorJson: {}, websiteUrl: str, translate: {}) -> None:
"""Sets a web address
"""
websiteUrl = websiteUrl.strip()
notUrl = False
if '.' not in websiteUrl:
notUrl = True
if '://' not in websiteUrl:
notUrl = True
if ' ' in websiteUrl:
notUrl = True
if '<' in websiteUrl:
notUrl = True
if not actorJson.get('attachment'):
actorJson['attachment'] = []
matchStrings = _getWebsiteStrings()
matchStrings.append(translate['Website'].lower())
# remove any existing value
propertyFound = None
for propertyValue in actorJson['attachment']:
if not propertyValue.get('name'):
continue
if not propertyValue.get('type'):
continue
if propertyValue['name'].lower() not in matchStrings:
continue
propertyFound = propertyValue
break
if propertyFound:
actorJson['attachment'].remove(propertyFound)
if notUrl:
return
newEntry = {
"name": 'Website',
"type": "PropertyValue",
"value": websiteUrl
}
actorJson['attachment'].append(newEntry)

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.4 KiB

View File

@ -519,6 +519,7 @@
"zorin": "zorin",
"solus": "solus",
"fedora": "fedora",
"redhat": "redhat",
"elementary": "elementary",
"prideflag": "pride",
"biflag": "biflag",
@ -671,6 +672,7 @@
"meownwn": "meownwn",
"meowderpy": "meowderpy",
"blobcat": "blobcat",
"blobthinksmart": "blobthinksmart",
"blobcathappy": "blobcathappy",
"blobcoolcat": "blobcoolcat",
"blobcatwink": "blobcatwink",
@ -766,5 +768,6 @@
"pine64": "pine64",
"void": "void",
"openbsd": "openbsd",
"freebsd": "freebsd"
"freebsd": "freebsd",
"orgmode": "orgmode"
}

BIN
emoji/orgmode.png 100644

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.0 KiB

BIN
emoji/redhat.png 100644

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

View File

@ -136,6 +136,8 @@
--containericons-horizontal-spacing: 1%;
--containericons-horizontal-spacing-mobile: 3%;
--containericons-horizontal-offset: -1%;
--containericons-vertical-align: 0.5%;
--containericons-vertical-align-mobile: 1%;
--likes-count-offset: 5px;
--likes-count-offset-mobile: 10px;
--publish-button-vertical-offset: 10px;
@ -1308,8 +1310,8 @@ div.container {
.containericons img {
float: var(--icons-side);
max-width: 200px;
width: 3%;
margin: 0px var(--containericons-horizontal-spacing);
width: 25px;
margin: var(--containericons-vertical-align) var(--containericons-horizontal-spacing);
margin-right: 0px;
border-radius: 0%;
}
@ -1512,7 +1514,7 @@ div.container {
color: var(--time-color);
margin: var(--time-vertical-align) 20px;
}
input[type=text], select, textarea {
input[type=text], input[type=password], select, textarea {
width: 100%;
padding: 12px;
border: 1px solid #ccc;
@ -1970,7 +1972,7 @@ div.container {
float: var(--icons-side);
max-width: 200px;
width: 7%;
margin: 1% var(--containericons-horizontal-spacing-mobile);
margin: var(--containericons-vertical-align-mobile) var(--containericons-horizontal-spacing-mobile);
margin-right: 0px;
border-radius: 0%;
}
@ -2166,7 +2168,7 @@ div.container {
color: var(--time-color);
margin: var(--time-vertical-align-mobile) 20px;
}
input[type=text], select, textarea {
input[type=text], input[type=password], select, textarea {
width: 100%;
padding: 12px;
border: 1px solid #ccc;

View File

@ -54,6 +54,8 @@ from follow import clearFollows
from follow import followerOfPerson
from follow import sendFollowRequestViaServer
from follow import sendUnfollowRequestViaServer
from tests import testSharedItemsFederation
from tests import testGroupFollow
from tests import testPostMessageBetweenServers
from tests import testFollowBetweenServers
from tests import testClientToServer
@ -85,6 +87,8 @@ from manualapprove import manualDenyFollowRequest
from manualapprove import manualApproveFollowRequest
from shares import sendShareViaServer
from shares import sendUndoShareViaServer
from shares import sendWantedViaServer
from shares import sendUndoWantedViaServer
from shares import addShare
from theme import setTheme
from announce import sendAnnounceViaServer
@ -110,6 +114,20 @@ parser = argparse.ArgumentParser(description='ActivityPub Server')
parser.add_argument('--userAgentBlocks', type=str,
default=None,
help='List of blocked user agents, separated by commas')
parser.add_argument('--libretranslate', dest='libretranslateUrl', type=str,
default=None,
help='URL for LibreTranslate service')
parser.add_argument('--conversationId', dest='conversationId', type=str,
default=None,
help='Conversation Id which can be added ' +
'when sending a post')
parser.add_argument('--libretranslateApiKey',
dest='libretranslateApiKey', type=str,
default=None,
help='API key for LibreTranslate service')
parser.add_argument('--defaultCurrency', dest='defaultCurrency', type=str,
default=None,
help='Default currency EUR/GBP/USD...')
parser.add_argument('-n', '--nickname', dest='nickname', type=str,
default=None,
help='Nickname of the account to use')
@ -257,6 +275,10 @@ parser.add_argument('--rss', dest='rss', type=str, default=None,
help='Show an rss feed for a given url')
parser.add_argument('-f', '--federate', nargs='+', dest='federationList',
help='Specify federation list separated by spaces')
parser.add_argument('--federateshares', nargs='+',
dest='sharedItemsFederatedDomains',
help='Specify federation list for shared items, ' +
'separated by spaces')
parser.add_argument("--following", "--followingList",
dest='followingList',
type=str2bool, nargs='?',
@ -309,6 +331,11 @@ parser.add_argument("--rssIconAtTop",
const=True, default=True,
help="Whether to show the rss icon at teh top or bottom" +
"of the timeline")
parser.add_argument("--lowBandwidth",
dest='lowBandwidth',
type=str2bool, nargs='?',
const=True, default=True,
help="Whether to use low bandwidth images")
parser.add_argument("--publishButtonAtTop",
dest='publishButtonAtTop',
type=str2bool, nargs='?',
@ -437,6 +464,9 @@ parser.add_argument('--minimumvotes', dest='minimumvotes', type=int,
default=1,
help='Minimum number of votes to remove or add' +
' a newswire item')
parser.add_argument('--maxLikeCount', dest='maxLikeCount', type=int,
default=10,
help='Maximum number of likes displayed on a post')
parser.add_argument('--votingtime', dest='votingtime', type=int,
default=1440,
help='Time to vote on newswire items in minutes')
@ -545,12 +575,27 @@ parser.add_argument('--itemName', dest='itemName', type=str,
parser.add_argument('--undoItemName', dest='undoItemName', type=str,
default=None,
help='Name of an shared item to remove')
parser.add_argument('--wantedItemName', dest='wantedItemName', type=str,
default=None,
help='Name of a wanted item')
parser.add_argument('--undoWantedItemName', dest='undoWantedItemName',
type=str, default=None,
help='Name of a wanted item to remove')
parser.add_argument('--summary', dest='summary', type=str,
default=None,
help='Description of an item being shared')
parser.add_argument('--itemImage', dest='itemImage', type=str,
default=None,
help='Filename of an image for an item being shared')
parser.add_argument('--itemQty', dest='itemQty', type=float,
default=1,
help='Quantity of items being shared')
parser.add_argument('--itemPrice', dest='itemPrice', type=str,
default="0.00",
help='Total price of items being shared')
parser.add_argument('--itemCurrency', dest='itemCurrency', type=str,
default="EUR",
help='Currency of items being shared')
parser.add_argument('--itemType', dest='itemType', type=str,
default=None,
help='Type of item being shared')
@ -588,6 +633,8 @@ if args.tests:
sys.exit()
if args.testsnetwork:
print('Network Tests')
testSharedItemsFederation()
testGroupFollow()
testPostMessageBetweenServers()
testFollowBetweenServers()
testClientToServer()
@ -606,6 +653,14 @@ if baseDir.endswith('/'):
print("--path option should not end with '/'")
sys.exit()
# automatic translations
if args.libretranslateUrl:
if '://' in args.libretranslateUrl and \
'.' in args.libretranslateUrl:
setConfigParam(baseDir, 'libretranslateUrl', args.libretranslateUrl)
if args.libretranslateApiKey:
setConfigParam(baseDir, 'libretranslateApiKey', args.libretranslateApiKey)
if args.posts:
if '@' not in args.posts:
if '/users/' in args.posts:
@ -631,9 +686,11 @@ if args.posts:
args.port = 80
elif args.gnunet:
proxyType = 'gnunet'
if not args.language:
args.language = 'en'
getPublicPostsOfPerson(baseDir, nickname, domain, False, True,
proxyType, args.port, httpPrefix, debug,
__version__)
__version__, args.language)
sys.exit()
if args.postDomains:
@ -663,12 +720,15 @@ if args.postDomains:
proxyType = 'gnunet'
wordFrequency = {}
domainList = []
if not args.language:
args.language = 'en'
domainList = getPublicPostDomains(None,
baseDir, nickname, domain,
proxyType, args.port,
httpPrefix, debug,
__version__,
wordFrequency, domainList)
wordFrequency, domainList,
args.language)
for postDomain in domainList:
print(postDomain)
sys.exit()
@ -703,12 +763,15 @@ if args.postDomainsBlocked:
proxyType = 'gnunet'
wordFrequency = {}
domainList = []
if not args.language:
args.language = 'en'
domainList = getPublicPostDomainsBlocked(None,
baseDir, nickname, domain,
proxyType, args.port,
httpPrefix, debug,
__version__,
wordFrequency, domainList)
wordFrequency, domainList,
args.language)
for postDomain in domainList:
print(postDomain)
sys.exit()
@ -741,12 +804,14 @@ if args.checkDomains:
elif args.gnunet:
proxyType = 'gnunet'
maxBlockedDomains = 0
if not args.language:
args.language = 'en'
checkDomains(None,
baseDir, nickname, domain,
proxyType, args.port,
httpPrefix, debug,
__version__,
maxBlockedDomains, False)
maxBlockedDomains, False, args.language)
sys.exit()
if args.socnet:
@ -758,10 +823,12 @@ if args.socnet:
if not args.http:
args.port = 443
proxyType = 'tor'
if not args.language:
args.language = 'en'
dotGraph = instancesGraph(baseDir, args.socnet,
proxyType, args.port,
httpPrefix, debug,
__version__)
__version__, args.language)
try:
with open('socnet.dot', 'w+') as fp:
fp.write(dotGraph)
@ -785,9 +852,11 @@ if args.postsraw:
proxyType = 'i2p'
elif args.gnunet:
proxyType = 'gnunet'
if not args.language:
args.language = 'en'
getPublicPostsOfPerson(baseDir, nickname, domain, False, False,
proxyType, args.port, httpPrefix, debug,
__version__)
__version__, args.language)
sys.exit()
if args.json:
@ -892,6 +961,8 @@ if not args.language:
languageCode = getConfigParam(baseDir, 'language')
if languageCode:
args.language = languageCode
else:
args.language = 'en'
# maximum number of new registrations
if not args.maxRegistrations:
@ -1095,7 +1166,6 @@ if args.message:
toDomain = 'public'
toPort = port
# ccUrl = httpPrefix + '://' + domain + '/users/' + nickname + '/followers'
ccUrl = None
sendMessage = args.message
followersOnly = args.followersonly
@ -1124,7 +1194,8 @@ if args.message:
args.commentsEnabled, attach, mediaType,
attachedImageDescription, city,
cachedWebfingers, personCache, isArticle,
args.debug, replyTo, replyTo, subject)
args.language, args.lowBandwidth, args.debug,
replyTo, replyTo, args.conversationId, subject)
for i in range(10):
# TODO detect send success/fail
time.sleep(1)
@ -1214,6 +1285,10 @@ if args.itemName:
'with the --summary option')
sys.exit()
if not args.itemQty:
print('Specify a quantity of shared items with the --itemQty option')
sys.exit()
if not args.itemType:
print('Specify a type of shared item with the --itemType option')
sys.exit()
@ -1224,7 +1299,7 @@ if args.itemName:
sys.exit()
if not args.location:
print('Specify a location or city where theshared ' +
print('Specify a location or city where the shared ' +
'item resides with the --location option')
sys.exit()
@ -1245,12 +1320,14 @@ if args.itemName:
args.itemName,
args.summary,
args.itemImage,
args.itemQty,
args.itemType,
args.itemCategory,
args.location,
args.duration,
cachedWebfingers, personCache,
debug, __version__)
debug, __version__,
args.itemPrice, args.itemCurrency)
for i in range(10):
# TODO detect send success/fail
time.sleep(1)
@ -1285,6 +1362,100 @@ if args.undoItemName:
time.sleep(1)
sys.exit()
if args.wantedItemName:
if not args.password:
args.password = getpass.getpass('Password: ')
if not args.password:
print('Specify a password with the --password option')
sys.exit()
args.password = args.password.replace('\n', '')
if not args.nickname:
print('Specify a nickname with the --nickname option')
sys.exit()
if not args.summary:
print('Specify a description for your shared item ' +
'with the --summary option')
sys.exit()
if not args.itemQty:
print('Specify a quantity of shared items with the --itemQty option')
sys.exit()
if not args.itemType:
print('Specify a type of shared item with the --itemType option')
sys.exit()
if not args.itemCategory:
print('Specify a category of shared item ' +
'with the --itemCategory option')
sys.exit()
if not args.location:
print('Specify a location or city where the wanted ' +
'item resides with the --location option')
sys.exit()
if not args.duration:
print('Specify a duration to share the object ' +
'with the --duration option')
sys.exit()
session = createSession(proxyType)
personCache = {}
cachedWebfingers = {}
print('Sending wanted item: ' + args.wantedItemName)
sendWantedViaServer(baseDir, session,
args.nickname, args.password,
domain, port,
httpPrefix,
args.wantedItemName,
args.summary,
args.itemImage,
args.itemQty,
args.itemType,
args.itemCategory,
args.location,
args.duration,
cachedWebfingers, personCache,
debug, __version__,
args.itemPrice, args.itemCurrency)
for i in range(10):
# TODO detect send success/fail
time.sleep(1)
sys.exit()
if args.undoWantedItemName:
if not args.password:
args.password = getpass.getpass('Password: ')
if not args.password:
print('Specify a password with the --password option')
sys.exit()
args.password = args.password.replace('\n', '')
if not args.nickname:
print('Specify a nickname with the --nickname option')
sys.exit()
session = createSession(proxyType)
personCache = {}
cachedWebfingers = {}
print('Sending undo of wanted item: ' + args.undoWantedItemName)
sendUndoWantedViaServer(baseDir, session,
args.nickname, args.password,
domain, port,
httpPrefix,
args.undoWantedItemName,
cachedWebfingers, personCache,
debug, __version__)
for i in range(10):
# TODO detect send success/fail
time.sleep(1)
sys.exit()
if args.like:
if not args.nickname:
print('Specify a nickname with the --nickname option')
@ -1674,6 +1845,10 @@ if args.followers:
nickname = args.followers.split('/u/')[1]
nickname = nickname.replace('\n', '').replace('\r', '')
domain = args.followers.split('/u/')[0]
elif '/c/' in args.followers:
nickname = args.followers.split('/c/')[1]
nickname = nickname.replace('\n', '').replace('\r', '')
domain = args.followers.split('/c/')[0]
else:
# format: @nick@domain
if '@' not in args.followers:
@ -1710,7 +1885,7 @@ if args.followers:
handle = nickname + '@' + domain
wfRequest = webfingerHandle(session, handle,
httpPrefix, cachedWebfingers,
None, __version__, debug)
None, __version__, debug, False)
if not wfRequest:
print('Unable to webfinger ' + handle)
sys.exit()
@ -1739,6 +1914,7 @@ if args.followers:
personUrl = personUrl.replace('/channel/', '/actor/')
personUrl = personUrl.replace('/profile/', '/actor/')
personUrl = personUrl.replace('/u/', '/actor/')
personUrl = personUrl.replace('/c/', '/actor/')
if not personUrl:
# try single user instance
personUrl = httpPrefix + '://' + domain
@ -1817,6 +1993,9 @@ if args.addgroup:
if not args.domain or not getConfigParam(baseDir, 'domain'):
print('Use the --domain option to set the domain name')
sys.exit()
if nickname.startswith('!'):
# remove preceding group indicator
nickname = nickname[1:]
if not validNickname(domain, nickname):
print(nickname + ' is a reserved name. Use something different.')
sys.exit()
@ -2090,11 +2269,27 @@ if args.desktop:
storeInboxPosts,
args.notifyShowNewPosts,
args.language,
args.debug)
args.debug, args.lowBandwidth)
sys.exit()
if federationList:
print('Federating with: ' + str(federationList))
if args.sharedItemsFederatedDomains:
print('Federating shared items with: ' +
args.sharedItemsFederatedDomains)
sharedItemsFederatedDomains = []
if args.sharedItemsFederatedDomains:
sharedItemsFederatedDomainsStr = args.sharedItemsFederatedDomains
setConfigParam(baseDir, 'sharedItemsFederatedDomains',
sharedItemsFederatedDomainsStr)
else:
sharedItemsFederatedDomainsStr = \
getConfigParam(baseDir, 'sharedItemsFederatedDomains')
if sharedItemsFederatedDomainsStr:
sharedItemsFederatedDomainsList = sharedItemsFederatedDomainsStr.split(',')
for sharedFederatedDomain in sharedItemsFederatedDomainsList:
sharedItemsFederatedDomains.append(sharedFederatedDomain.strip())
if args.block:
if not nickname:
@ -2245,6 +2440,7 @@ if args.unfilterStr:
sys.exit()
if args.testdata:
args.language = 'en'
city = 'London, England'
nickname = 'testuser567'
password = 'boringpassword'
@ -2287,21 +2483,21 @@ if args.testdata:
"spanner",
"It's a spanner",
"img/shares1.png",
"tool",
1, "tool",
"mechanical",
"City",
"City", "0", "GBP",
"2 months",
debug, city)
debug, city, args.language, {}, 'shares', args.lowBandwidth)
addShare(baseDir,
httpPrefix, nickname, domain, port,
"witch hat",
"Spooky",
"img/shares2.png",
"hat",
1, "hat",
"clothing",
"City",
"City", "0", "GBP",
"3 months",
debug, city)
debug, city, args.language, {}, 'shares', args.lowBandwidth)
deleteAllPosts(baseDir, nickname, domain, 'inbox')
deleteAllPosts(baseDir, nickname, domain, 'outbox')
@ -2322,6 +2518,8 @@ if args.testdata:
testEventTime = None
testLocation = None
testIsArticle = False
conversationId = None
lowBandwidth = False
createPublicPost(baseDir, nickname, domain, port, httpPrefix,
"like this is totally just a #test man",
@ -2334,7 +2532,8 @@ if args.testdata:
testInReplyTo, testInReplyToAtomUri,
testSubject, testSchedulePost,
testEventDate, testEventTime, testLocation,
testIsArticle)
testIsArticle, args.language, conversationId,
lowBandwidth)
createPublicPost(baseDir, nickname, domain, port, httpPrefix,
"Zoiks!!!",
testFollowersOnly,
@ -2346,7 +2545,8 @@ if args.testdata:
testInReplyTo, testInReplyToAtomUri,
testSubject, testSchedulePost,
testEventDate, testEventTime, testLocation,
testIsArticle)
testIsArticle, args.language, conversationId,
lowBandwidth)
createPublicPost(baseDir, nickname, domain, port, httpPrefix,
"Hey scoob we need like a hundred more #milkshakes",
testFollowersOnly,
@ -2358,7 +2558,8 @@ if args.testdata:
testInReplyTo, testInReplyToAtomUri,
testSubject, testSchedulePost,
testEventDate, testEventTime, testLocation,
testIsArticle)
testIsArticle, args.language, conversationId,
lowBandwidth)
createPublicPost(baseDir, nickname, domain, port, httpPrefix,
"Getting kinda spooky around here",
testFollowersOnly,
@ -2370,7 +2571,8 @@ if args.testdata:
'someone', testInReplyToAtomUri,
testSubject, testSchedulePost,
testEventDate, testEventTime, testLocation,
testIsArticle)
testIsArticle, args.language, conversationId,
lowBandwidth)
createPublicPost(baseDir, nickname, domain, port, httpPrefix,
"And they would have gotten away with it too" +
"if it wasn't for those pesky hackers",
@ -2383,7 +2585,8 @@ if args.testdata:
testInReplyTo, testInReplyToAtomUri,
testSubject, testSchedulePost,
testEventDate, testEventTime, testLocation,
testIsArticle)
testIsArticle, args.language, conversationId,
lowBandwidth)
createPublicPost(baseDir, nickname, domain, port, httpPrefix,
"man these centralized sites are like the worst!",
testFollowersOnly,
@ -2395,7 +2598,8 @@ if args.testdata:
testInReplyTo, testInReplyToAtomUri,
testSubject, testSchedulePost,
testEventDate, testEventTime, testLocation,
testIsArticle)
testIsArticle, args.language, conversationId,
lowBandwidth)
createPublicPost(baseDir, nickname, domain, port, httpPrefix,
"another mystery solved #test",
testFollowersOnly,
@ -2407,7 +2611,8 @@ if args.testdata:
testInReplyTo, testInReplyToAtomUri,
testSubject, testSchedulePost,
testEventDate, testEventTime, testLocation,
testIsArticle)
testIsArticle, args.language, conversationId,
lowBandwidth)
createPublicPost(baseDir, nickname, domain, port, httpPrefix,
"let's go bowling",
testFollowersOnly,
@ -2419,21 +2624,22 @@ if args.testdata:
testInReplyTo, testInReplyToAtomUri,
testSubject, testSchedulePost,
testEventDate, testEventTime, testLocation,
testIsArticle)
testIsArticle, args.language, conversationId,
lowBandwidth)
domainFull = domain + ':' + str(port)
clearFollows(baseDir, nickname, domain)
followPerson(baseDir, nickname, domain, 'maxboardroom', domainFull,
federationList, False)
federationList, False, False)
followPerson(baseDir, nickname, domain, 'ultrapancake', domainFull,
federationList, False)
federationList, False, False)
followPerson(baseDir, nickname, domain, 'sausagedog', domainFull,
federationList, False)
federationList, False, False)
followPerson(baseDir, nickname, domain, 'drokk', domainFull,
federationList, False)
federationList, False, False)
followerOfPerson(baseDir, nickname, domain, 'drokk', domainFull,
federationList, False)
federationList, False, False)
followerOfPerson(baseDir, nickname, domain, 'maxboardroom', domainFull,
federationList, False)
federationList, False, False)
setConfigParam(baseDir, 'admin', nickname)
# set a lower bound to the maximum mentions
@ -2506,6 +2712,11 @@ sendThreadsTimeoutMins = \
if sendThreadsTimeoutMins is not None:
args.sendThreadsTimeoutMins = int(sendThreadsTimeoutMins)
maxLikeCount = \
getConfigParam(baseDir, 'maxLikeCount')
if maxLikeCount is not None:
args.maxLikeCount = int(maxLikeCount)
showPublishAsIcon = \
getConfigParam(baseDir, 'showPublishAsIcon')
if showPublishAsIcon is not None:
@ -2561,6 +2772,11 @@ showNodeInfoVersion = \
if showNodeInfoVersion is not None:
args.showNodeInfoVersion = bool(showNodeInfoVersion)
lowBandwidth = \
getConfigParam(baseDir, 'lowBandwidth')
if lowBandwidth is not None:
args.lowBandwidth = bool(lowBandwidth)
userAgentsBlocked = []
if args.userAgentBlocks:
userAgentsBlockedStr = args.userAgentBlocks
@ -2608,8 +2824,18 @@ if args.registration:
setConfigParam(baseDir, 'registration', 'closed')
print('New registrations closed')
defaultCurrency = getConfigParam(baseDir, 'defaultCurrency')
if not defaultCurrency:
setConfigParam(baseDir, 'defaultCurrency', 'EUR')
if args.defaultCurrency:
if args.defaultCurrency == args.defaultCurrency.upper():
setConfigParam(baseDir, 'defaultCurrency', args.defaultCurrency)
print('Default currency set to ' + args.defaultCurrency)
if __name__ == "__main__":
runDaemon(userAgentsBlocked,
runDaemon(args.lowBandwidth, args.maxLikeCount,
sharedItemsFederatedDomains,
userAgentsBlocked,
args.logLoginFailures,
args.city,
args.showNodeInfoAccounts,

View File

@ -119,14 +119,22 @@ def _isFilteredBase(filename: str, content: str) -> bool:
return False
def isFilteredGlobally(baseDir: str, content: str) -> bool:
"""Is the given content globally filtered?
"""
globalFiltersFilename = baseDir + '/accounts/filters.txt'
if _isFilteredBase(globalFiltersFilename, content):
return True
return False
def isFiltered(baseDir: str, nickname: str, domain: str, content: str) -> bool:
"""Should the given content be filtered out?
This is a simple type of filter which just matches words, not a regex
You can add individual words or use word1+word2 to indicate that two
words must be present although not necessarily adjacent
"""
globalFiltersFilename = baseDir + '/accounts/filters.txt'
if _isFilteredBase(globalFiltersFilename, content):
if isFilteredGlobally(baseDir, content):
return True
if not nickname or not domain:

211
follow.py
View File

@ -28,12 +28,16 @@ from utils import saveJson
from utils import isAccountDir
from utils import getUserPaths
from utils import acctDir
from utils import hasGroupType
from utils import isGroupAccount
from utils import localActorUrl
from acceptreject import createAccept
from acceptreject import createReject
from webfinger import webfingerHandle
from auth import createBasicAuthHeader
from session import getJson
from session import postJson
from cache import getPersonPubKey
def createInitialLastSeen(baseDir: str, httpPrefix: str) -> None:
@ -62,11 +66,11 @@ def createInitialLastSeen(baseDir: str, httpPrefix: str) -> None:
handle = handle.replace('\n', '')
nickname = handle.split('@')[0]
domain = handle.split('@')[1]
actor = \
httpPrefix + '://' + domain + '/users/' + nickname
if nickname.startswith('!'):
nickname = nickname[1:]
actor = localActorUrl(httpPrefix, nickname, domain)
lastSeenFilename = \
lastSeenDir + '/' + actor.replace('/', '#') + '.txt'
print('lastSeenFilename: ' + lastSeenFilename)
if not os.path.isfile(lastSeenFilename):
with open(lastSeenFilename, 'w+') as fp:
fp.write(str(100))
@ -108,12 +112,11 @@ def _removeFromFollowBase(baseDir: str,
acceptDenyDomain = acceptOrDenyHandle.split('@')[1]
# for each possible users path construct an actor and
# check if it exists in teh file
usersPaths = ('users', 'profile', 'channel', 'accounts', 'u')
usersPaths = getUserPaths()
actorFound = False
for usersName in usersPaths:
acceptDenyActor = \
'://' + acceptDenyDomain + '/' + \
usersName + '/' + acceptDenyNickname
'://' + acceptDenyDomain + usersName + acceptDenyNickname
if acceptDenyActor in open(approveFollowsFilename).read():
actorFound = True
break
@ -195,12 +198,13 @@ def getMutualsOfPerson(baseDir: str,
def followerOfPerson(baseDir: str, nickname: str, domain: str,
followerNickname: str, followerDomain: str,
federationList: [], debug: bool) -> bool:
federationList: [], debug: bool,
groupAccount: bool) -> bool:
"""Adds a follower of the given person
"""
return followPerson(baseDir, nickname, domain,
followerNickname, followerDomain,
federationList, debug, 'followers.txt')
federationList, debug, groupAccount, 'followers.txt')
def isFollowerOfPerson(baseDir: str, nickname: str, domain: str,
@ -234,13 +238,15 @@ def isFollowerOfPerson(baseDir: str, nickname: str, domain: str,
def unfollowAccount(baseDir: str, nickname: str, domain: str,
followNickname: str, followDomain: str,
followFile: str = 'following.txt',
debug: bool = False) -> bool:
debug: bool, groupAccount: bool,
followFile: str = 'following.txt') -> bool:
"""Removes a person to the follow list
"""
domain = removeDomainPort(domain)
handle = nickname + '@' + domain
handleToUnfollow = followNickname + '@' + followDomain
if groupAccount:
handleToUnfollow = '!' + handleToUnfollow
if not os.path.isdir(baseDir + '/accounts'):
os.mkdir(baseDir + '/accounts')
if not os.path.isdir(baseDir + '/accounts/' + handle):
@ -261,8 +267,9 @@ def unfollowAccount(baseDir: str, nickname: str, domain: str,
lines = f.readlines()
with open(filename, 'w+') as f:
for line in lines:
if line.strip("\n").strip("\r").lower() != \
handleToUnfollowLower:
checkHandle = line.strip("\n").strip("\r").lower()
if checkHandle != handleToUnfollowLower and \
checkHandle != '!' + handleToUnfollowLower:
f.write(line)
# write to an unfollowed file so that if a follow accept
@ -282,16 +289,16 @@ def unfollowAccount(baseDir: str, nickname: str, domain: str,
def unfollowerOfAccount(baseDir: str, nickname: str, domain: str,
followerNickname: str, followerDomain: str,
debug: bool = False) -> bool:
debug: bool, groupAccount: bool) -> bool:
"""Remove a follower of a person
"""
return unfollowAccount(baseDir, nickname, domain,
followerNickname, followerDomain,
'followers.txt', debug)
debug, groupAccount, 'followers.txt')
def clearFollows(baseDir: str, nickname: str, domain: str,
followFile='following.txt') -> None:
followFile: str = 'following.txt') -> None:
"""Removes all follows
"""
handle = nickname + '@' + domain
@ -392,11 +399,10 @@ def getFollowingFeed(baseDir: str, domain: str, port: int, path: str,
if headerOnly:
firstStr = \
httpPrefix + '://' + domain + '/users/' + \
nickname + '/' + followFile + '?page=1'
localActorUrl(httpPrefix, nickname, domain) + \
'/' + followFile + '?page=1'
idStr = \
httpPrefix + '://' + domain + '/users/' + \
nickname + '/' + followFile
localActorUrl(httpPrefix, nickname, domain) + '/' + followFile
totalStr = \
_getNoOfFollows(baseDir, nickname, domain, authorized)
following = {
@ -413,10 +419,10 @@ def getFollowingFeed(baseDir: str, domain: str, port: int, path: str,
nextPageNumber = int(pageNumber + 1)
idStr = \
httpPrefix + '://' + domain + '/users/' + \
nickname + '/' + followFile + '?page=' + str(pageNumber)
localActorUrl(httpPrefix, nickname, domain) + \
'/' + followFile + '?page=' + str(pageNumber)
partOfStr = \
httpPrefix + '://' + domain + '/users/' + nickname + '/' + followFile
localActorUrl(httpPrefix, nickname, domain) + '/' + followFile
following = {
'@context': 'https://www.w3.org/ns/activitystreams',
'id': idStr,
@ -446,10 +452,14 @@ def getFollowingFeed(baseDir: str, domain: str, port: int, path: str,
if currPage == pageNumber:
line2 = \
line.lower().replace('\n', '').replace('\r', '')
url = httpPrefix + '://' + \
line2.split('@')[1] + \
'/users/' + \
line2.split('@')[0]
nick = line2.split('@')[0]
dom = line2.split('@')[1]
if not nick.startswith('!'):
# person actor
url = localActorUrl(httpPrefix, nick, dom)
else:
# group actor
url = httpPrefix + '://' + dom + '/c/' + nick
following['orderedItems'].append(url)
elif ((line.startswith('http') or
line.startswith('hyper')) and
@ -470,8 +480,8 @@ def getFollowingFeed(baseDir: str, domain: str, port: int, path: str,
lastPage = 1
if nextPageNumber > lastPage:
following['next'] = \
httpPrefix + '://' + domain + '/users/' + \
nickname + '/' + followFile + '?page=' + str(lastPage)
localActorUrl(httpPrefix, nickname, domain) + \
'/' + followFile + '?page=' + str(lastPage)
return following
@ -535,7 +545,8 @@ def _storeFollowRequest(baseDir: str,
nicknameToFollow: str, domainToFollow: str, port: int,
nickname: str, domain: str, fromPort: int,
followJson: {},
debug: bool, personUrl: str) -> bool:
debug: bool, personUrl: str,
groupAccount: bool) -> bool:
"""Stores the follow request for later use
"""
accountsDir = baseDir + '/accounts/' + \
@ -543,10 +554,12 @@ def _storeFollowRequest(baseDir: str,
if not os.path.isdir(accountsDir):
return False
approveHandle = nickname + '@' + domain
domainFull = getFullDomain(domain, fromPort)
approveHandle = getFullDomain(nickname + '@' + domain, fromPort)
if groupAccount:
approveHandle = '!' + approveHandle
followersFilename = accountsDir + '/followers.txt'
if os.path.isfile(followersFilename):
alreadyFollowing = False
@ -557,14 +570,13 @@ def _storeFollowRequest(baseDir: str,
if approveHandle in followersStr:
alreadyFollowing = True
elif '://' + domainFull + '/profile/' + nickname in followersStr:
alreadyFollowing = True
elif '://' + domainFull + '/channel/' + nickname in followersStr:
alreadyFollowing = True
elif '://' + domainFull + '/accounts/' + nickname in followersStr:
alreadyFollowing = True
elif '://' + domainFull + '/u/' + nickname in followersStr:
alreadyFollowing = True
else:
usersPaths = getUserPaths()
for possibleUsersPath in usersPaths:
url = '://' + domainFull + possibleUsersPath + nickname
if url in followersStr:
alreadyFollowing = True
break
if alreadyFollowing:
if debug:
@ -590,6 +602,8 @@ def _storeFollowRequest(baseDir: str,
approveHandleStored = approveHandle
if '/users/' not in personUrl:
approveHandleStored = personUrl
if groupAccount:
approveHandle = '!' + approveHandle
if os.path.isfile(approveFollowsFilename):
if approveHandle not in open(approveFollowsFilename).read():
@ -617,7 +631,7 @@ def receiveFollowRequest(session, baseDir: str, httpPrefix: str,
cachedWebfingers: {}, personCache: {},
messageJson: {}, federationList: [],
debug: bool, projectVersion: str,
maxFollowers: int) -> bool:
maxFollowers: int, onionDomain: str) -> bool:
"""Receives a follow request within the POST section of HTTPServer
"""
if not messageJson['type'].startswith('Follow'):
@ -723,36 +737,78 @@ def receiveFollowRequest(session, baseDir: str, httpPrefix: str,
print('Too many follow requests')
return False
# Get the actor for the follower and add it to the cache.
# Getting their public key has the same result
if debug:
print('Obtaining the following actor: ' + messageJson['actor'])
if not getPersonPubKey(baseDir, session, messageJson['actor'],
personCache, debug, projectVersion,
httpPrefix, domainToFollow, onionDomain):
if debug:
print('Unable to obtain following actor: ' +
messageJson['actor'])
groupAccount = \
hasGroupType(baseDir, messageJson['actor'], personCache)
if groupAccount and isGroupAccount(baseDir, nickname, domain):
print('Group cannot follow a group')
return False
print('Storing follow request for approval')
return _storeFollowRequest(baseDir,
nicknameToFollow, domainToFollow, port,
nickname, domain, fromPort,
messageJson, debug, messageJson['actor'])
messageJson, debug, messageJson['actor'],
groupAccount)
else:
print('Follow request does not require approval')
print('Follow request does not require approval ' + approveHandle)
# update the followers
if os.path.isdir(baseDir + '/accounts/' +
nicknameToFollow + '@' + domainToFollow):
followersFilename = \
baseDir + '/accounts/' + \
nicknameToFollow + '@' + domainToFollow + '/followers.txt'
accountToBeFollowed = \
acctDir(baseDir, nicknameToFollow, domainToFollow)
if os.path.isdir(accountToBeFollowed):
followersFilename = accountToBeFollowed + '/followers.txt'
# for actors which don't follow the mastodon
# /users/ path convention store the full actor
if '/users/' not in messageJson['actor']:
approveHandle = messageJson['actor']
# Get the actor for the follower and add it to the cache.
# Getting their public key has the same result
if debug:
print('Obtaining the following actor: ' + messageJson['actor'])
if not getPersonPubKey(baseDir, session, messageJson['actor'],
personCache, debug, projectVersion,
httpPrefix, domainToFollow, onionDomain):
if debug:
print('Unable to obtain following actor: ' +
messageJson['actor'])
print('Updating followers file: ' +
followersFilename + ' adding ' + approveHandle)
if os.path.isfile(followersFilename):
if approveHandle not in open(followersFilename).read():
groupAccount = \
hasGroupType(baseDir,
messageJson['actor'], personCache)
if debug:
print(approveHandle + ' / ' + messageJson['actor'] +
' is Group: ' + str(groupAccount))
if groupAccount and \
isGroupAccount(baseDir, nickname, domain):
print('Group cannot follow a group')
return False
try:
with open(followersFilename, 'r+') as followersFile:
content = followersFile.read()
if approveHandle + '\n' not in content:
followersFile.seek(0, 0)
followersFile.write(approveHandle + '\n' +
content)
if not groupAccount:
followersFile.write(approveHandle +
'\n' + content)
else:
followersFile.write('!' + approveHandle +
'\n' + content)
except Exception as e:
print('WARN: ' +
'Failed to write entry to followers file ' +
@ -815,13 +871,20 @@ def followedAccountAccepts(session, baseDir: str, httpPrefix: str,
except BaseException:
pass
groupAccount = False
if followJson:
if followJson.get('actor'):
if hasGroupType(baseDir, followJson['actor'], personCache):
groupAccount = True
return sendSignedJson(acceptJson, session, baseDir,
nicknameToFollow, domainToFollow, port,
nickname, domain, fromPort, '',
httpPrefix, True, clientToServer,
federationList,
sendThreads, postLog, cachedWebfingers,
personCache, debug, projectVersion)
personCache, debug, projectVersion, None,
groupAccount)
def followedAccountRejects(session, baseDir: str, httpPrefix: str,
@ -867,6 +930,9 @@ def followedAccountRejects(session, baseDir: str, httpPrefix: str,
nickname + '@' + domain + ' port ' + str(fromPort))
clientToServer = False
denyHandle = getFullDomain(nickname + '@' + domain, fromPort)
groupAccount = False
if hasGroupType(baseDir, personUrl, personCache):
groupAccount = True
# remove from the follow requests file
removeFromFollowRequests(baseDir, nicknameToFollow, domainToFollow,
denyHandle, debug)
@ -882,7 +948,8 @@ def followedAccountRejects(session, baseDir: str, httpPrefix: str,
httpPrefix, True, clientToServer,
federationList,
sendThreads, postLog, cachedWebfingers,
personCache, debug, projectVersion)
personCache, debug, projectVersion, None,
groupAccount)
def sendFollowRequest(session, baseDir: str,
@ -901,15 +968,20 @@ def sendFollowRequest(session, baseDir: str,
return None
fullDomain = getFullDomain(domain, port)
followActor = httpPrefix + '://' + fullDomain + '/users/' + nickname
followActor = localActorUrl(httpPrefix, nickname, fullDomain)
requestDomain = getFullDomain(followDomain, followPort)
statusNumber, published = getStatusNumber()
groupAccount = False
if followNickname:
followedId = followedActor
followHandle = followNickname + '@' + requestDomain
groupAccount = hasGroupType(baseDir, followedActor, personCache)
if groupAccount:
followHandle = '!' + followHandle
print('Follow request being sent to group account')
else:
if debug:
print('DEBUG: sendFollowRequest - assuming single user instance')
@ -924,6 +996,9 @@ def sendFollowRequest(session, baseDir: str,
'actor': followActor,
'object': followedId
}
if groupAccount:
newFollowJson['to'] = followedId
print('Follow request: ' + str(newFollowJson))
if _followApprovalRequired(baseDir, nickname, domain, debug,
followHandle):
@ -941,7 +1016,7 @@ def sendFollowRequest(session, baseDir: str,
httpPrefix, True, clientToServer,
federationList,
sendThreads, postLog, cachedWebfingers, personCache,
debug, projectVersion)
debug, projectVersion, None, groupAccount)
return newFollowJson
@ -964,10 +1039,9 @@ def sendFollowRequestViaServer(baseDir: str, session,
followDomainFull = getFullDomain(followDomain, followPort)
followActor = httpPrefix + '://' + \
fromDomainFull + '/users/' + fromNickname
followedId = httpPrefix + '://' + \
followDomainFull + '/users/' + followNickname
followActor = localActorUrl(httpPrefix, fromNickname, fromDomainFull)
followedId = \
httpPrefix + '://' + followDomainFull + '/@' + followNickname
statusNumber, published = getStatusNumber()
newFollowJson = {
@ -983,7 +1057,7 @@ def sendFollowRequestViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = \
webfingerHandle(session, handle, httpPrefix, cachedWebfingers,
fromDomain, projectVersion, debug)
fromDomain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: follow request webfinger failed for ' + handle)
@ -1050,10 +1124,9 @@ def sendUnfollowRequestViaServer(baseDir: str, session,
fromDomainFull = getFullDomain(fromDomain, fromPort)
followDomainFull = getFullDomain(followDomain, followPort)
followActor = httpPrefix + '://' + \
fromDomainFull + '/users/' + fromNickname
followedId = httpPrefix + '://' + \
followDomainFull + '/users/' + followNickname
followActor = localActorUrl(httpPrefix, fromNickname, fromDomainFull)
followedId = \
httpPrefix + '://' + followDomainFull + '/@' + followNickname
statusNumber, published = getStatusNumber()
unfollowJson = {
@ -1074,7 +1147,7 @@ def sendUnfollowRequestViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = \
webfingerHandle(session, handle, httpPrefix, cachedWebfingers,
fromDomain, projectVersion, debug)
fromDomain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: unfollow webfinger failed for ' + handle)
@ -1140,7 +1213,7 @@ def getFollowingViaServer(baseDir: str, session,
return 6
domainFull = getFullDomain(domain, port)
followActor = httpPrefix + '://' + domainFull + '/users/' + nickname
followActor = localActorUrl(httpPrefix, nickname, domainFull)
authHeader = createBasicAuthHeader(nickname, password)
@ -1181,7 +1254,7 @@ def getFollowersViaServer(baseDir: str, session,
return 6
domainFull = getFullDomain(domain, port)
followActor = httpPrefix + '://' + domainFull + '/users/' + nickname
followActor = localActorUrl(httpPrefix, nickname, domainFull)
authHeader = createBasicAuthHeader(nickname, password)
@ -1222,7 +1295,7 @@ def getFollowRequestsViaServer(baseDir: str, session,
domainFull = getFullDomain(domain, port)
followActor = httpPrefix + '://' + domainFull + '/users/' + nickname
followActor = localActorUrl(httpPrefix, nickname, domainFull)
authHeader = createBasicAuthHeader(nickname, password)
headers = {
@ -1263,7 +1336,7 @@ def approveFollowRequestViaServer(baseDir: str, session,
return 6
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
authHeader = createBasicAuthHeader(nickname, password)
@ -1303,7 +1376,7 @@ def denyFollowRequestViaServer(baseDir: str, session,
return 6
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
authHeader = createBasicAuthHeader(nickname, password)
@ -1417,8 +1490,10 @@ def outboxUndoFollow(baseDir: str, messageJson: {}, debug: bool) -> None:
getDomainFromActor(messageJson['object']['object'])
domainFollowingFull = getFullDomain(domainFollowing, portFollowing)
groupAccount = hasGroupType(baseDir, messageJson['object']['object'], None)
if unfollowAccount(baseDir, nicknameFollower, domainFollowerFull,
nicknameFollowing, domainFollowingFull):
nicknameFollowing, domainFollowingFull,
debug, groupAccount):
if debug:
print('DEBUG: ' + nicknameFollower + ' unfollowed ' +
nicknameFollowing + '@' + domainFollowingFull)

View File

@ -13,6 +13,7 @@ import os
def _dirAcct(baseDir: str, nickname: str, domain: str) -> str:
return baseDir + '/accounts/' + nickname + '@' + domain
def _portDomainRemove(domain: str) -> str:
"""If the domain has a port appended then remove it
eg. mydomain.com:80 becomes mydomain.com

View File

@ -21,8 +21,8 @@ Epicyon is written in Python with a HTML+CSS web interface and uses no javascrip
Emojis, hashtags, photos, video and audio attachments, instance and account level blocking controls, moderation functions and reports are all supported. Build the community you want and avoid the stuff you don't. No ads. No blockchains or other Silicon Valley garbage.
=> https://epicyon.net/epicyon.tar.gz Download
=> https://epicyon.net/#install Install
=> https://libreserver.org/epicyon/epicyon.tar.gz Download
=> https://libreserver.org/epicyon/#install Install
=> https://gitlab.com/bashrc2/epicyon Repo
=> https://www.patreon.com/freedombone Donate
=> features.gmi Features

View File

@ -55,6 +55,9 @@ def saveEventPost(baseDir: str, handle: str, postId: str,
See https://framagit.org/framasoft/mobilizon/-/blob/
master/lib/federation/activity_stream/converter/event.ex
"""
if not os.path.isdir(baseDir + '/accounts/' + handle):
print('WARN: Account does not exist at ' +
baseDir + '/accounts/' + handle)
calendarPath = baseDir + '/accounts/' + handle + '/calendar'
if not os.path.isdir(calendarPath):
os.mkdir(calendarPath)

View File

@ -24,6 +24,7 @@ from time import gmtime, strftime
import datetime
from utils import getFullDomain
from utils import getSHA256
from utils import localActorUrl
def messageContentDigest(messageBodyJsonStr: str) -> str:
@ -48,7 +49,7 @@ def signPostHeaders(dateStr: str, privateKeyPem: str,
if not dateStr:
dateStr = strftime("%a, %d %b %Y %H:%M:%S %Z", gmtime())
keyID = httpPrefix + '://' + domain + '/users/' + nickname + '#main-key'
keyID = localActorUrl(httpPrefix, nickname, domain) + '#main-key'
if not messageBodyJsonStr:
headers = {
'(request-target)': f'post {path}',
@ -125,7 +126,7 @@ def signPostHeadersNew(dateStr: str, privateKeyPem: str,
currTime = datetime.datetime.strptime(dateStr, timeFormat)
secondsSinceEpoch = \
int((currTime - datetime.datetime(1970, 1, 1)).total_seconds())
keyID = httpPrefix + '://' + domain + '/users/' + nickname + '#main-key'
keyID = localActorUrl(httpPrefix, nickname, domain) + '#main-key'
if not messageBodyJsonStr:
headers = {
'*request-target': f'post {path}',

396
inbox.py
View File

@ -13,6 +13,9 @@ import datetime
import time
import random
from linked_data_sig import verifyJsonSignature
from languages import understoodPostLanguage
from utils import getUserPaths
from utils import getBaseContentFromPost
from utils import acctDir
from utils import removeDomainPort
from utils import getPortFromDomain
@ -23,7 +26,6 @@ from utils import getConfigParam
from utils import hasUsersPath
from utils import validPostDate
from utils import getFullDomain
from utils import isEventPost
from utils import removeIdEnding
from utils import getProtocolPrefixes
from utils import isBlogPost
@ -43,18 +45,19 @@ from utils import loadJson
from utils import saveJson
from utils import updateLikesCollection
from utils import undoLikesCollectionEntry
from utils import hasGroupType
from utils import localActorUrl
from categories import getHashtagCategories
from categories import setHashtagCategory
from httpsig import verifyPostHeaders
from session import createSession
from session import getJson
from follow import isFollowingActor
from follow import receiveFollowRequest
from follow import getFollowersOfActor
from follow import unfollowerOfAccount
from pprint import pprint
from cache import getPersonFromCache
from cache import storePersonInCache
from cache import getPersonPubKey
from acceptreject import receiveAcceptReject
from bookmarks import updateBookmarksCollection
from bookmarks import undoBookmarksCollectionEntry
@ -87,7 +90,9 @@ from categories import guessHashtagCategory
from context import hasValidContext
from speaker import updateSpeaker
from announce import isSelfAnnounce
from announce import createAnnounce
from notifyOnPost import notifyWhenPersonPosts
from conversation import updateConversation
def storeHashTags(baseDir: str, nickname: str, postJsonObject: {}) -> None:
@ -151,7 +156,7 @@ def storeHashTags(baseDir: str, nickname: str, postJsonObject: {}) -> None:
categoryStr = \
guessHashtagCategory(tagName, hashtagCategories)
if categoryStr:
setHashtagCategory(baseDir, tagName, categoryStr)
setHashtagCategory(baseDir, tagName, categoryStr, False)
def _inboxStorePostToHtmlCache(recentPostsCache: {}, maxRecentPosts: int,
@ -164,7 +169,8 @@ def _inboxStorePostToHtmlCache(recentPostsCache: {}, maxRecentPosts: int,
showPublishedDateOnly: bool,
peertubeInstances: [],
allowLocalNetworkAccess: bool,
themeName: str) -> None:
themeName: str, systemLanguage: str,
maxLikeCount: int) -> None:
"""Converts the json post into html and stores it in a cache
This enables the post to be quickly displayed later
"""
@ -182,7 +188,7 @@ def _inboxStorePostToHtmlCache(recentPostsCache: {}, maxRecentPosts: int,
httpPrefix, __version__, boxname, None,
showPublishedDateOnly,
peertubeInstances, allowLocalNetworkAccess,
themeName,
themeName, systemLanguage, maxLikeCount,
not isDM(postJsonObject),
True, True, False, True)
@ -215,67 +221,25 @@ def validInboxFilenames(baseDir: str, nickname: str, domain: str,
domain = removeDomainPort(domain)
inboxDir = acctDir(baseDir, nickname, domain) + '/inbox'
if not os.path.isdir(inboxDir):
print('Not an inbox directory: ' + inboxDir)
return True
expectedStr = expectedDomain + ':' + str(expectedPort)
expectedFound = False
for subdir, dirs, files in os.walk(inboxDir):
for f in files:
filename = os.path.join(subdir, f)
if not os.path.isfile(filename):
print('filename: ' + filename)
return False
if expectedStr not in filename:
print('Expected: ' + expectedStr)
print('Invalid filename: ' + filename)
return False
if expectedStr in filename:
expectedFound = True
break
if not expectedFound:
print('Expected file was not found: ' + expectedStr)
return False
return True
def getPersonPubKey(baseDir: str, session, personUrl: str,
personCache: {}, debug: bool,
projectVersion: str, httpPrefix: str,
domain: str, onionDomain: str) -> str:
if not personUrl:
return None
personUrl = personUrl.replace('#main-key', '')
if personUrl.endswith('/users/inbox'):
if debug:
print('DEBUG: Obtaining public key for shared inbox')
personUrl = personUrl.replace('/users/inbox', '/inbox')
personJson = \
getPersonFromCache(baseDir, personUrl, personCache, True)
if not personJson:
if debug:
print('DEBUG: Obtaining public key for ' + personUrl)
personDomain = domain
if onionDomain:
if '.onion/' in personUrl:
personDomain = onionDomain
profileStr = 'https://www.w3.org/ns/activitystreams'
asHeader = {
'Accept': 'application/activity+json; profile="' + profileStr + '"'
}
personJson = \
getJson(session, personUrl, asHeader, None, debug,
projectVersion, httpPrefix, personDomain)
if not personJson:
return None
pubKey = None
if personJson.get('publicKey'):
if personJson['publicKey'].get('publicKeyPem'):
pubKey = personJson['publicKey']['publicKeyPem']
else:
if personJson.get('publicKeyPem'):
pubKey = personJson['publicKeyPem']
if not pubKey:
if debug:
print('DEBUG: Public key not found for ' + personUrl)
storePersonInCache(baseDir, personUrl, personJson, personCache, True)
return pubKey
def inboxMessageHasParams(messageJson: {}) -> bool:
"""Checks whether an incoming message contains expected parameters
"""
@ -351,8 +315,8 @@ def savePostToInboxQueue(baseDir: str, httpPrefix: str,
messageBytes: str,
httpHeaders: {},
postPath: str, debug: bool,
blockedCache: []) -> str:
"""Saves the give json to the inbox queue for the person
blockedCache: [], systemLanguage: str) -> str:
"""Saves the given json to the inbox queue for the person
keyId specifies the actor sending the post
"""
if len(messageBytes) > 10240:
@ -415,9 +379,9 @@ def savePostToInboxQueue(baseDir: str, httpPrefix: str,
replyNickname + '@' + replyDomain)
return None
if postJsonObject['object'].get('content'):
if isinstance(postJsonObject['object']['content'], str):
if isFiltered(baseDir, nickname, domain,
postJsonObject['object']['content']):
contentStr = getBaseContentFromPost(postJsonObject, systemLanguage)
if contentStr:
if isFiltered(baseDir, nickname, domain, contentStr):
if debug:
print('WARN: post was filtered out due to content')
return None
@ -438,8 +402,8 @@ def savePostToInboxQueue(baseDir: str, httpPrefix: str,
if actor:
postId = actor + '/statuses/' + statusNumber
else:
postId = httpPrefix + '://' + originalDomain + \
'/users/' + nickname + '/statuses/' + statusNumber
postId = localActorUrl(httpPrefix, nickname, originalDomain) + \
'/statuses/' + statusNumber
# NOTE: don't change postJsonObject['id'] before signature check
@ -669,10 +633,11 @@ def _receiveUndoFollow(session, baseDir: str, httpPrefix: str,
getDomainFromActor(messageJson['object']['object'])
domainFollowingFull = getFullDomain(domainFollowing, portFollowing)
groupAccount = hasGroupType(baseDir, messageJson['object']['actor'], None)
if unfollowerOfAccount(baseDir,
nicknameFollowing, domainFollowingFull,
nicknameFollower, domainFollowerFull,
debug):
debug, groupAccount):
print(nicknameFollowing + '@' + domainFollowingFull + ': '
'Follower ' + nicknameFollower + '@' + domainFollowerFull +
' was removed')
@ -712,6 +677,11 @@ def _receiveUndo(session, baseDir: str, httpPrefix: str,
if debug:
print('DEBUG: ' + messageJson['type'] + ' has no object type')
return False
if not isinstance(messageJson['object']['type'], str):
if debug:
print('DEBUG: ' + messageJson['type'] +
' type within object is not a string')
return False
if not messageJson['object'].get('object'):
if debug:
print('DEBUG: ' + messageJson['type'] +
@ -730,25 +700,6 @@ def _receiveUndo(session, baseDir: str, httpPrefix: str,
return False
def _receiveEventPost(recentPostsCache: {}, session, baseDir: str,
httpPrefix: str, domain: str, port: int,
sendThreads: [], postLog: [], cachedWebfingers: {},
personCache: {}, messageJson: {}, federationList: [],
nickname: str, debug: bool) -> bool:
"""Receive a mobilizon-type event activity
See https://framagit.org/framasoft/mobilizon/-/blob/
master/lib/federation/activity_stream/converter/event.ex
"""
if not isEventPost(messageJson):
return
print('Receiving event: ' + str(messageJson['object']))
handle = getFullDomain(nickname + '@' + domain, port)
postId = removeIdEnding(messageJson['id']).replace('/', '#')
saveEventPost(baseDir, handle, postId, messageJson['object'])
def _personReceiveUpdate(baseDir: str,
domain: str, port: int,
updateNickname: str, updateDomain: str,
@ -762,10 +713,10 @@ def _personReceiveUpdate(baseDir: str,
' ' + str(personJson))
domainFull = getFullDomain(domain, port)
updateDomainFull = getFullDomain(updateDomain, updatePort)
usersPaths = ('users', 'profile', 'channel', 'accounts', 'u')
usersPaths = getUserPaths()
usersStrFound = False
for usersStr in usersPaths:
actor = updateDomainFull + '/' + usersStr + '/' + updateNickname
actor = updateDomainFull + usersStr + updateNickname
if actor in personJson['id']:
usersStrFound = True
break
@ -883,6 +834,11 @@ def _receiveUpdate(recentPostsCache: {}, session, baseDir: str,
if debug:
print('DEBUG: ' + messageJson['type'] + ' object has no type')
return False
if not isinstance(messageJson['object']['type'], str):
if debug:
print('DEBUG: ' + messageJson['type'] +
' object type is not string')
return False
if not hasUsersPath(messageJson['actor']):
if debug:
print('DEBUG: "users" or "profile" missing from actor in ' +
@ -1305,7 +1261,7 @@ def _receiveAnnounce(recentPostsCache: {},
debug: bool, translate: {},
YTReplacementDomain: str,
allowLocalNetworkAccess: bool,
themeName: str) -> bool:
themeName: str, systemLanguage: str) -> bool:
"""Receives an announce activity within the POST section of HTTPServer
"""
if messageJson['type'] != 'Announce':
@ -1385,6 +1341,7 @@ def _receiveAnnounce(recentPostsCache: {},
if debug:
print('DEBUG: Downloading announce post ' + messageJson['actor'] +
' -> ' + messageJson['object'])
domainFull = getFullDomain(domain, port)
postJsonObject = downloadAnnounce(session, baseDir,
httpPrefix,
nickname, domain,
@ -1392,7 +1349,9 @@ def _receiveAnnounce(recentPostsCache: {},
__version__, translate,
YTReplacementDomain,
allowLocalNetworkAccess,
recentPostsCache, debug)
recentPostsCache, debug,
systemLanguage,
domainFull, personCache)
if not postJsonObject:
notInOnion = True
if onionDomain:
@ -1616,7 +1575,10 @@ def _estimateNumberOfEmoji(content: str) -> int:
def _validPostContent(baseDir: str, nickname: str, domain: str,
messageJson: {}, maxMentions: int, maxEmoji: int,
allowLocalNetworkAccess: bool, debug: bool) -> bool:
allowLocalNetworkAccess: bool, debug: bool,
systemLanguage: str,
httpPrefix: str, domainFull: str,
personCache: {}) -> bool:
"""Is the content of a received post valid?
Check for bad html
Check for hellthreads
@ -1651,27 +1613,27 @@ def _validPostContent(baseDir: str, nickname: str, domain: str,
messageJson['object']['content']):
return True
if dangerousMarkup(messageJson['object']['content'],
allowLocalNetworkAccess):
contentStr = getBaseContentFromPost(messageJson, systemLanguage)
if dangerousMarkup(contentStr, allowLocalNetworkAccess):
if messageJson['object'].get('id'):
print('REJECT ARBITRARY HTML: ' + messageJson['object']['id'])
print('REJECT ARBITRARY HTML: bad string in post - ' +
messageJson['object']['content'])
contentStr)
return False
# check (rough) number of mentions
mentionsEst = _estimateNumberOfMentions(messageJson['object']['content'])
mentionsEst = _estimateNumberOfMentions(contentStr)
if mentionsEst > maxMentions:
if messageJson['object'].get('id'):
print('REJECT HELLTHREAD: ' + messageJson['object']['id'])
print('REJECT HELLTHREAD: Too many mentions in post - ' +
messageJson['object']['content'])
contentStr)
return False
if _estimateNumberOfEmoji(messageJson['object']['content']) > maxEmoji:
if _estimateNumberOfEmoji(contentStr) > maxEmoji:
if messageJson['object'].get('id'):
print('REJECT EMOJI OVERLOAD: ' + messageJson['object']['id'])
print('REJECT EMOJI OVERLOAD: Too many emoji in post - ' +
messageJson['object']['content'])
contentStr)
return False
# check number of tags
if messageJson['object'].get('tag'):
@ -1684,9 +1646,14 @@ def _validPostContent(baseDir: str, nickname: str, domain: str,
print('REJECT: Too many tags in post - ' +
messageJson['object']['tag'])
return False
# check that the post is in a language suitable for this account
if not understoodPostLanguage(baseDir, nickname, domain,
messageJson, systemLanguage,
httpPrefix, domainFull,
personCache):
return False
# check for filtered content
if isFiltered(baseDir, nickname, domain,
messageJson['object']['content']):
if isFiltered(baseDir, nickname, domain, contentStr):
print('REJECT: content filtered')
return False
if messageJson['object'].get('inReplyTo'):
@ -1910,90 +1877,69 @@ def _groupHandle(baseDir: str, handle: str) -> bool:
return actorJson['type'] == 'Group'
def _getGroupName(baseDir: str, handle: str) -> str:
"""Returns the preferred name of a group
"""
actorFile = baseDir + '/accounts/' + handle + '.json'
if not os.path.isfile(actorFile):
return False
actorJson = loadJson(actorFile)
if not actorJson:
return 'Group'
return actorJson['name']
def _sendToGroupMembers(session, baseDir: str, handle: str, port: int,
postJsonObject: {},
httpPrefix: str, federationList: [],
sendThreads: [], postLog: [], cachedWebfingers: {},
personCache: {}, debug: bool) -> None:
personCache: {}, debug: bool,
systemLanguage: str,
onionDomain: str, i2pDomain: str) -> None:
"""When a post arrives for a group send it out to the group members
"""
if debug:
print('\n\n=========================================================')
print(handle + ' sending to group members')
sharedItemFederationTokens = {}
sharedItemsFederatedDomains = []
sharedItemsFederatedDomainsStr = \
getConfigParam(baseDir, 'sharedItemsFederatedDomains')
if sharedItemsFederatedDomainsStr:
siFederatedDomainsList = \
sharedItemsFederatedDomainsStr.split(',')
for sharedFederatedDomain in siFederatedDomainsList:
domainStr = sharedFederatedDomain.strip()
sharedItemsFederatedDomains.append(domainStr)
followersFile = baseDir + '/accounts/' + handle + '/followers.txt'
if not os.path.isfile(followersFile):
return
if not postJsonObject.get('to'):
return
if not postJsonObject.get('object'):
return
nickname = handle.split('@')[0]
# groupname = _getGroupName(baseDir, handle)
if not hasObjectDict(postJsonObject):
return
nickname = handle.split('@')[0].replace('!', '')
domain = handle.split('@')[1]
domainFull = getFullDomain(domain, port)
# set sender
groupActor = localActorUrl(httpPrefix, nickname, domainFull)
if groupActor not in postJsonObject['to']:
return
cc = ''
sendingActor = postJsonObject['actor']
sendingActorNickname = getNicknameFromActor(sendingActor)
sendingActorDomain, sendingActorPort = \
getDomainFromActor(sendingActor)
sendingActorDomainFull = \
getFullDomain(sendingActorDomain, sendingActorPort)
senderStr = '@' + sendingActorNickname + '@' + sendingActorDomainFull
if not postJsonObject['object']['content'].startswith(senderStr):
postJsonObject['object']['content'] = \
senderStr + ' ' + postJsonObject['object']['content']
# add mention to tag list
if not postJsonObject['object']['tag']:
postJsonObject['object']['tag'] = []
# check if the mention already exists
mentionExists = False
for mention in postJsonObject['object']['tag']:
if mention['type'] == 'Mention':
if mention.get('href'):
if mention['href'] == sendingActor:
mentionExists = True
if not mentionExists:
# add the mention of the original sender
postJsonObject['object']['tag'].append({
'href': sendingActor,
'name': senderStr,
'type': 'Mention'
})
nickname = handle.split('@')[0].replace('!', '')
postJsonObject['actor'] = \
httpPrefix + '://' + domainFull + '/users/' + nickname
postJsonObject['to'] = \
[postJsonObject['actor'] + '/followers']
postJsonObject['cc'] = [cc]
postJsonObject['object']['to'] = postJsonObject['to']
postJsonObject['object']['cc'] = [cc]
# set subject
if not postJsonObject['object'].get('summary'):
postJsonObject['object']['summary'] = 'General Discussion'
domain = removeDomainPort(domain)
with open(followersFile, 'r') as groupMembers:
for memberHandle in groupMembers:
if memberHandle != handle:
memberNickname = memberHandle.split('@')[0]
memberDomain = memberHandle.split('@')[1]
memberPort = port
if ':' in memberDomain:
memberPort = getPortFromDomain(memberDomain)
memberDomain = removeDomainPort(memberDomain)
sendSignedJson(postJsonObject, session, baseDir,
nickname, domain, port,
memberNickname, memberDomain, memberPort, cc,
httpPrefix, False, False, federationList,
sendThreads, postLog, cachedWebfingers,
personCache, debug, __version__)
if debug:
print('Group announce: ' + postJsonObject['object']['id'])
announceJson = \
createAnnounce(session, baseDir, federationList,
nickname, domain, port,
groupActor + '/followers', cc,
httpPrefix,
postJsonObject['object']['id'],
False, False,
sendThreads, postLog,
personCache, cachedWebfingers,
debug, __version__)
sendToFollowersThread(session, baseDir, nickname, domain,
onionDomain, i2pDomain, port,
httpPrefix, federationList,
sendThreads, postLog,
cachedWebfingers, personCache,
announceJson, debug, __version__,
sharedItemsFederatedDomains,
sharedItemFederationTokens)
def _inboxUpdateCalendar(baseDir: str, handle: str,
@ -2107,7 +2053,7 @@ def _bounceDM(senderPostId: str, session, httpPrefix: str,
sendThreads: [], postLog: [],
cachedWebfingers: {}, personCache: {},
translate: {}, debug: bool,
lastBounceMessage: []) -> bool:
lastBounceMessage: [], systemLanguage: str) -> bool:
"""Sends a bounce message back to the sending handle
if a DM has been rejected
"""
@ -2126,6 +2072,10 @@ def _bounceDM(senderPostId: str, session, httpPrefix: str,
lastBounceMessage[0] = currTime
senderNickname = sendingHandle.split('@')[0]
groupAccount = False
if sendingHandle.startswith('!'):
sendingHandle = sendingHandle[1:]
groupAccount = True
senderDomain = sendingHandle.split('@')[1]
senderPort = port
if ':' in senderDomain:
@ -2150,6 +2100,8 @@ def _bounceDM(senderPostId: str, session, httpPrefix: str,
eventDate = None
eventTime = None
location = None
conversationId = None
lowBandwidth = False
postJsonObject = \
createDirectMessagePost(baseDir, nickname, domain, port,
httpPrefix, content, followersOnly,
@ -2159,7 +2111,8 @@ def _bounceDM(senderPostId: str, session, httpPrefix: str,
imageDescription, city,
inReplyTo, inReplyToAtomUri,
subject, debug, schedulePost,
eventDate, eventTime, location)
eventDate, eventTime, location,
systemLanguage, conversationId, lowBandwidth)
if not postJsonObject:
print('WARN: unable to create bounce message to ' + sendingHandle)
return False
@ -2170,7 +2123,7 @@ def _bounceDM(senderPostId: str, session, httpPrefix: str,
senderNickname, senderDomain, senderPort, cc,
httpPrefix, False, False, federationList,
sendThreads, postLog, cachedWebfingers,
personCache, debug, __version__)
personCache, debug, __version__, None, groupAccount)
return True
@ -2183,7 +2136,7 @@ def _isValidDM(baseDir: str, nickname: str, domain: str, port: int,
personCache: {},
translate: {}, debug: bool,
lastBounceMessage: [],
handle: str) -> bool:
handle: str, systemLanguage: str) -> bool:
"""Is the given message a valid DM?
"""
if nickname == 'inbox':
@ -2196,8 +2149,8 @@ def _isValidDM(baseDir: str, nickname: str, domain: str, port: int,
if not os.path.isfile(followDMsFilename):
# dm index will be updated
updateIndexList.append('dm')
_dmNotify(baseDir, handle,
httpPrefix + '://' + domain + '/users/' + nickname + '/dm')
actUrl = localActorUrl(httpPrefix, nickname, domain)
_dmNotify(baseDir, handle, actUrl + '/dm')
return True
# get the file containing following handles
@ -2258,13 +2211,14 @@ def _isValidDM(baseDir: str, nickname: str, domain: str, port: int,
cachedWebfingers,
personCache,
translate, debug,
lastBounceMessage)
lastBounceMessage,
systemLanguage)
return False
# dm index will be updated
updateIndexList.append('dm')
_dmNotify(baseDir, handle,
httpPrefix + '://' + domain + '/users/' + nickname + '/dm')
actUrl = localActorUrl(httpPrefix, nickname, domain)
_dmNotify(baseDir, handle, actUrl + '/dm')
return True
@ -2284,7 +2238,8 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
allowLocalNetworkAccess: bool,
peertubeInstances: [],
lastBounceMessage: [],
themeName: str) -> bool:
themeName: str, systemLanguage: str,
maxLikeCount: int) -> bool:
""" Anything which needs to be done after initial checks have passed
"""
actor = keyId
@ -2365,7 +2320,7 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
debug, translate,
YTReplacementDomain,
allowLocalNetworkAccess,
themeName):
themeName, systemLanguage):
if debug:
print('DEBUG: Announce accepted from ' + actor)
@ -2412,9 +2367,12 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
nickname = handle.split('@')[0]
jsonObj = None
domainFull = getFullDomain(domain, port)
if _validPostContent(baseDir, nickname, domain,
postJsonObject, maxMentions, maxEmoji,
allowLocalNetworkAccess, debug):
allowLocalNetworkAccess, debug,
systemLanguage, httpPrefix,
domainFull, personCache):
if postJsonObject.get('object'):
jsonObj = postJsonObject['object']
@ -2447,7 +2405,7 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
return False
# replace YouTube links, so they get less tracking data
replaceYouTube(postJsonObject, YTReplacementDomain)
replaceYouTube(postJsonObject, YTReplacementDomain, systemLanguage)
# list of indexes to be updated
updateIndexList = ['inbox']
@ -2464,6 +2422,20 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
# if the votes on a question have changed then
# send out an update
questionJson['type'] = 'Update'
sharedItemsFederatedDomains = []
sharedItemFederationTokens = {}
sharedItemFederationTokens = {}
sharedItemsFederatedDomains = []
sharedItemsFederatedDomainsStr = \
getConfigParam(baseDir, 'sharedItemsFederatedDomains')
if sharedItemsFederatedDomainsStr:
siFederatedDomainsList = \
sharedItemsFederatedDomainsStr.split(',')
for sharedFederatedDomain in siFederatedDomainsList:
domainStr = sharedFederatedDomain.strip()
sharedItemsFederatedDomains.append(domainStr)
sendToFollowersThread(session, baseDir,
nickname, domain,
onionDomain, i2pDomain, port,
@ -2471,7 +2443,9 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
sendThreads, postLog,
cachedWebfingers, personCache,
postJsonObject, debug,
__version__)
__version__,
sharedItemsFederatedDomains,
sharedItemFederationTokens)
isReplyToMutedPost = False
@ -2488,28 +2462,34 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
personCache,
translate, debug,
lastBounceMessage,
handle):
handle, systemLanguage):
return False
# get the actor being replied to
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
# create a reply notification file if needed
if not postIsDM and isReply(postJsonObject, actor):
if nickname != 'inbox':
# replies index will be updated
updateIndexList.append('tlreplies')
conversationId = None
if postJsonObject['object'].get('conversation'):
conversationId = \
postJsonObject['object']['conversation']
if postJsonObject['object'].get('inReplyTo'):
inReplyTo = postJsonObject['object']['inReplyTo']
if inReplyTo:
if isinstance(inReplyTo, str):
if not isMuted(baseDir, nickname, domain,
inReplyTo):
inReplyTo, conversationId):
actUrl = \
localActorUrl(httpPrefix,
nickname, domain)
_replyNotify(baseDir, handle,
httpPrefix + '://' + domain +
'/users/' + nickname +
'/tlreplies')
actUrl + '/tlreplies')
else:
isReplyToMutedPost = True
@ -2517,7 +2497,8 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
nickname, domain, postJsonObject,
translate, YTReplacementDomain,
allowLocalNetworkAccess,
recentPostsCache, debug):
recentPostsCache, debug, systemLanguage,
domainFull, personCache):
# media index will be updated
updateIndexList.append('tlmedia')
if isBlogPost(postJsonObject):
@ -2544,10 +2525,10 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
if notifyWhenPersonPosts(baseDir, nickname, domain,
fromNickname, fromDomainFull):
postId = removeIdEnding(jsonObj['id'])
domFull = getFullDomain(domain, port)
postLink = \
httpPrefix + '://' + \
getFullDomain(domain, port) + \
'/users/' + nickname + \
localActorUrl(httpPrefix,
nickname, domFull) + \
'?notifypost=' + postId.replace('/', '-')
_notifyPostArrival(baseDir, handle, postLink)
@ -2591,7 +2572,8 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
showPublishedDateOnly,
peertubeInstances,
allowLocalNetworkAccess,
themeName)
themeName, systemLanguage,
maxLikeCount)
if debug:
timeDiff = \
str(int((time.time() - htmlCacheStartTime) *
@ -2600,9 +2582,11 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
' post as html to cache in ' +
timeDiff + ' mS')
handleName = handle.split('@')[0]
updateConversation(baseDir, handleName, domain, postJsonObject)
_inboxUpdateCalendar(baseDir, handle, postJsonObject)
handleName = handle.split('@')[0]
storeHashTags(baseDir, handleName, postJsonObject)
# send the post out to group members
@ -2611,7 +2595,8 @@ def _inboxAfterInitial(recentPostsCache: {}, maxRecentPosts: int,
postJsonObject,
httpPrefix, federationList, sendThreads,
postLog, cachedWebfingers, personCache,
debug)
debug, systemLanguage,
onionDomain, i2pDomain)
# if the post wasn't saved
if not os.path.isfile(destinationFilename):
@ -2850,7 +2835,8 @@ def runInboxQueue(recentPostsCache: {}, maxRecentPosts: int,
maxFollowers: int, allowLocalNetworkAccess: bool,
peertubeInstances: [],
verifyAllSignatures: bool,
themeName: str) -> None:
themeName: str, systemLanguage: str,
maxLikeCount: int) -> None:
"""Processes received items and moves them to the appropriate
directories
"""
@ -3122,7 +3108,7 @@ def runInboxQueue(recentPostsCache: {}, maxRecentPosts: int,
queueJson['post'],
federationList,
debug, projectVersion,
maxFollowers):
maxFollowers, onionDomain):
if os.path.isfile(queueFilename):
os.remove(queueFilename)
if len(queue) > 0:
@ -3147,23 +3133,6 @@ def runInboxQueue(recentPostsCache: {}, maxRecentPosts: int,
queue.pop(0)
continue
if _receiveEventPost(recentPostsCache, session,
baseDir, httpPrefix,
domain, port,
sendThreads, postLog,
cachedWebfingers,
personCache,
queueJson['post'],
federationList,
queueJson['postNickname'],
debug):
print('Queue: Event activity accepted from ' + keyId)
if os.path.isfile(queueFilename):
os.remove(queueFilename)
if len(queue) > 0:
queue.pop(0)
continue
if _receiveUpdate(recentPostsCache, session,
baseDir, httpPrefix,
domain, port,
@ -3255,7 +3224,8 @@ def runInboxQueue(recentPostsCache: {}, maxRecentPosts: int,
allowLocalNetworkAccess,
peertubeInstances,
lastBounceMessage,
themeName)
themeName, systemLanguage,
maxLikeCount)
if debug:
pprint(queueJson['post'])
print('Queue: Queue post accepted')

309
languages.py 100644
View File

@ -0,0 +1,309 @@
__filename__ = "languages.py"
__author__ = "Bob Mottram"
__license__ = "AGPL3+"
__version__ = "1.2.0"
__maintainer__ = "Bob Mottram"
__email__ = "bob@freedombone.net"
__status__ = "Production"
__module_group__ = "Core"
import os
import json
from urllib import request, parse
from utils import getActorLanguagesList
from utils import removeHtml
from utils import hasObjectDict
from utils import getConfigParam
from utils import localActorUrl
from cache import getPersonFromCache
def getActorLanguages(actorJson: {}) -> str:
"""Returns a string containing languages used by the given actor
"""
langList = getActorLanguagesList(actorJson)
if not langList:
return ''
languagesStr = ''
for lang in langList:
if languagesStr:
languagesStr += ' / ' + lang
else:
languagesStr = lang
return languagesStr
def setActorLanguages(baseDir: str, actorJson: {}, languagesStr: str) -> None:
"""Sets the languages used by the given actor
"""
separator = ','
if '/' in languagesStr:
separator = '/'
elif ',' in languagesStr:
separator = ','
elif ';' in languagesStr:
separator = ';'
elif '+' in languagesStr:
separator = '+'
elif ' ' in languagesStr:
separator = ' '
langList = languagesStr.lower().split(separator)
langList2 = ''
for lang in langList:
lang = lang.strip()
if baseDir:
languageFilename = baseDir + '/translations/' + lang + '.json'
if os.path.isfile(languageFilename):
if langList2:
langList2 += ', ' + lang.strip()
else:
langList2 += lang.strip()
else:
if langList2:
langList2 += ', ' + lang.strip()
else:
langList2 += lang.strip()
# remove any existing value
propertyFound = None
for propertyValue in actorJson['attachment']:
if not propertyValue.get('name'):
continue
if not propertyValue.get('type'):
continue
if not propertyValue['name'].lower().startswith('languages'):
continue
propertyFound = propertyValue
break
if propertyFound:
actorJson['attachment'].remove(propertyFound)
if not langList2:
return
newLanguages = {
"name": "Languages",
"type": "PropertyValue",
"value": langList2
}
actorJson['attachment'].append(newLanguages)
def understoodPostLanguage(baseDir: str, nickname: str, domain: str,
messageJson: {}, systemLanguage: str,
httpPrefix: str, domainFull: str,
personCache: {}) -> bool:
"""Returns true if the post is written in a language
understood by this account
"""
msgObject = messageJson
if hasObjectDict(messageJson):
msgObject = messageJson['object']
if not msgObject.get('contentMap'):
return True
if not isinstance(msgObject['contentMap'], dict):
return True
if msgObject['contentMap'].get(systemLanguage):
return True
personUrl = localActorUrl(httpPrefix, nickname, domainFull)
actorJson = getPersonFromCache(baseDir, personUrl, personCache, False)
if not actorJson:
print('WARN: unable to load actor to check languages ' + personUrl)
return False
languagesUnderstood = getActorLanguagesList(actorJson)
if not languagesUnderstood:
return True
for lang in languagesUnderstood:
if msgObject['contentMap'].get(lang):
return True
# is the language for this post supported by libretranslate?
libretranslateUrl = getConfigParam(baseDir, "libretranslateUrl")
if libretranslateUrl:
libretranslateApiKey = getConfigParam(baseDir, "libretranslateApiKey")
langList = \
libretranslateLanguages(libretranslateUrl, libretranslateApiKey)
for lang in langList:
if msgObject['contentMap'].get(lang):
return True
return False
def libretranslateLanguages(url: str, apiKey: str = None) -> []:
"""Returns a list of supported languages
"""
if not url:
return []
if not url.endswith('/languages'):
if not url.endswith('/'):
url += "/languages"
else:
url += "languages"
params = dict()
if apiKey:
params["api_key"] = apiKey
urlParams = parse.urlencode(params)
req = request.Request(url, data=urlParams.encode())
response = request.urlopen(req)
response_str = response.read().decode()
result = json.loads(response_str)
if not result:
return []
if not isinstance(result, list):
return []
langList = []
for lang in result:
if not isinstance(lang, dict):
continue
if not lang.get('code'):
continue
langCode = lang['code']
if len(langCode) != 2:
continue
langList.append(langCode)
langList.sort()
return langList
def getLinksFromContent(content: str) -> {}:
"""Returns a list of links within the given content
"""
if '<a href' not in content:
return {}
sections = content.split('<a href')
first = True
links = {}
for subsection in sections:
if first:
first = False
continue
if '"' not in subsection:
continue
url = subsection.split('"')[1].strip()
if '://' in url and '.' in url and \
'>' in subsection:
if url not in links:
linkText = subsection.split('>')[1]
if '<' in linkText:
linkText = linkText.split('<')[0]
links[linkText] = url
return links
def addLinksToContent(content: str, links: {}) -> str:
"""Adds links back into plain text
"""
for linkText, url in links.items():
urlDesc = url
if linkText.startswith('@') and linkText in content:
content = \
content.replace(linkText,
'<a href="' + url +
'" rel="nofollow noopener ' +
'noreferrer" target="_blank">' +
linkText + '</a>')
else:
if len(urlDesc) > 40:
urlDesc = urlDesc[:40]
content += \
'<p><a href="' + url + \
'" rel="nofollow noopener noreferrer" target="_blank">' + \
urlDesc + '</a></p>'
return content
def libretranslate(url: str, text: str,
source: str, target: str, apiKey: str = None) -> str:
"""Translate string using libretranslate
"""
if not url:
return None
if not url.endswith('/translate'):
if not url.endswith('/'):
url += "/translate"
else:
url += "translate"
originalText = text
# get any links from the text
links = getLinksFromContent(text)
# LibreTranslate doesn't like markup
text = removeHtml(text)
# remove any links from plain text version of the content
for _, url in links.items():
text = text.replace(url, '')
ltParams = {
"q": text,
"source": source,
"target": target
}
if apiKey:
ltParams["api_key"] = apiKey
urlParams = parse.urlencode(ltParams)
req = request.Request(url, data=urlParams.encode())
try:
response = request.urlopen(req)
except BaseException:
print('Unable to translate: ' + text)
return originalText
response_str = response.read().decode()
translatedText = \
'<p>' + json.loads(response_str)['translatedText'] + '</p>'
# append links form the original text
if links:
translatedText = addLinksToContent(translatedText, links)
return translatedText
def autoTranslatePost(baseDir: str, postJsonObject: {},
systemLanguage: str, translate: {}) -> str:
"""Tries to automatically translate the given post
"""
if not hasObjectDict(postJsonObject):
return ''
msgObject = postJsonObject['object']
if not msgObject.get('contentMap'):
return ''
if not isinstance(msgObject['contentMap'], dict):
return ''
# is the language for this post supported by libretranslate?
libretranslateUrl = getConfigParam(baseDir, "libretranslateUrl")
if not libretranslateUrl:
return ''
libretranslateApiKey = getConfigParam(baseDir, "libretranslateApiKey")
langList = \
libretranslateLanguages(libretranslateUrl, libretranslateApiKey)
for lang in langList:
if msgObject['contentMap'].get(lang):
content = msgObject['contentMap'][lang]
translatedText = \
libretranslate(libretranslateUrl, content,
lang, systemLanguage,
libretranslateApiKey)
if translatedText:
if removeHtml(translatedText) == removeHtml(content):
return content
translatedText = \
'<p>' + translate['Translated'].upper() + '</p>' + \
translatedText
return translatedText
return ''

23
like.py
View File

@ -18,6 +18,8 @@ from utils import getDomainFromActor
from utils import locatePost
from utils import updateLikesCollection
from utils import undoLikesCollectionEntry
from utils import hasGroupType
from utils import localActorUrl
from posts import sendSignedJson
from session import postJson
from webfinger import webfingerHandle
@ -74,7 +76,7 @@ def _like(recentPostsCache: {},
newLikeJson = {
"@context": "https://www.w3.org/ns/activitystreams",
'type': 'Like',
'actor': httpPrefix + '://' + fullDomain + '/users/' + nickname,
'actor': localActorUrl(httpPrefix, nickname, fullDomain),
'object': objectUrl
}
if ccList:
@ -85,13 +87,20 @@ def _like(recentPostsCache: {},
likedPostNickname = None
likedPostDomain = None
likedPostPort = None
groupAccount = False
if actorLiked:
likedPostNickname = getNicknameFromActor(actorLiked)
likedPostDomain, likedPostPort = getDomainFromActor(actorLiked)
groupAccount = hasGroupType(baseDir, actorLiked, personCache)
else:
if hasUsersPath(objectUrl):
likedPostNickname = getNicknameFromActor(objectUrl)
likedPostDomain, likedPostPort = getDomainFromActor(objectUrl)
if '/' + str(likedPostNickname) + '/' in objectUrl:
actorLiked = \
objectUrl.split('/' + likedPostNickname + '/')[0] + \
'/' + likedPostNickname
groupAccount = hasGroupType(baseDir, actorLiked, personCache)
if likedPostNickname:
postFilename = locatePost(baseDir, nickname, domain, objectUrl)
@ -113,7 +122,7 @@ def _like(recentPostsCache: {},
'https://www.w3.org/ns/activitystreams#Public',
httpPrefix, True, clientToServer, federationList,
sendThreads, postLog, cachedWebfingers, personCache,
debug, projectVersion)
debug, projectVersion, None, groupAccount)
return newLikeJson
@ -131,7 +140,7 @@ def likePost(recentPostsCache: {},
"""
likeDomain = getFullDomain(likeDomain, likePort)
actorLiked = httpPrefix + '://' + likeDomain + '/users/' + likeNickname
actorLiked = localActorUrl(httpPrefix, likeNickname, likeDomain)
objectUrl = actorLiked + '/statuses/' + str(likeStatusNumber)
return _like(recentPostsCache,
@ -155,7 +164,7 @@ def sendLikeViaServer(baseDir: str, session,
fromDomainFull = getFullDomain(fromDomain, fromPort)
actor = httpPrefix + '://' + fromDomainFull + '/users/' + fromNickname
actor = localActorUrl(httpPrefix, fromNickname, fromDomainFull)
newLikeJson = {
"@context": "https://www.w3.org/ns/activitystreams",
@ -169,7 +178,7 @@ def sendLikeViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
fromDomain, projectVersion, debug)
fromDomain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: like webfinger failed for ' + handle)
@ -233,7 +242,7 @@ def sendUndoLikeViaServer(baseDir: str, session,
fromDomainFull = getFullDomain(fromDomain, fromPort)
actor = httpPrefix + '://' + fromDomainFull + '/users/' + fromNickname
actor = localActorUrl(httpPrefix, fromNickname, fromDomainFull)
newUndoLikeJson = {
"@context": "https://www.w3.org/ns/activitystreams",
@ -251,7 +260,7 @@ def sendUndoLikeViaServer(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
fromDomain, projectVersion, debug)
fromDomain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: unlike webfinger failed for ' + handle)

View File

@ -109,7 +109,10 @@ def manualApproveFollowRequest(session, baseDir: str,
if approveHandle in approveFollowsStr:
exists = True
elif '@' in approveHandle:
reqNick = approveHandle.split('@')[0]
groupAccount = False
if approveHandle.startswith('!'):
groupAccount = True
reqNick = approveHandle.split('@')[0].replace('!', '')
reqDomain = approveHandle.split('@')[1].strip()
reqPrefix = httpPrefix + '://' + reqDomain
paths = getUserPaths()
@ -117,6 +120,8 @@ def manualApproveFollowRequest(session, baseDir: str,
if reqPrefix + userPath + reqNick in approveFollowsStr:
exists = True
approveHandleFull = reqPrefix + userPath + reqNick
if groupAccount:
approveHandleFull = '!' + approveHandleFull
break
if not exists:
print('Manual follow accept: ' + approveHandleFull +

View File

@ -140,6 +140,9 @@ def mastoApiV1Response(path: str, callingDomain: str,
_getMastoApiV1Account(baseDir, pathNickname, domain)
sendJsonStr = 'masto API account sent for ' + nickname
# NOTE: adding support for '/api/v1/directory seems to create
# federation problems, so avoid implementing that
if path.startswith('/api/v1/blocks'):
sendJson = []
sendJsonStr = 'masto API instance blocks sent'

View File

@ -8,11 +8,13 @@ __status__ = "Production"
__module_group__ = "Timeline"
import os
import time
import datetime
import subprocess
from random import randint
from hashlib import sha1
from auth import createPassword
from utils import getBaseContentFromPost
from utils import getFullDomain
from utils import getImageExtensions
from utils import getVideoExtensions
@ -26,7 +28,8 @@ from shutil import move
from city import spoofGeolocation
def replaceYouTube(postJsonObject: {}, replacementDomain: str) -> None:
def replaceYouTube(postJsonObject: {}, replacementDomain: str,
systemLanguage: str) -> None:
"""Replace YouTube with a replacement domain
This denies Google some, but not all, tracking data
"""
@ -36,11 +39,13 @@ def replaceYouTube(postJsonObject: {}, replacementDomain: str) -> None:
return
if not postJsonObject['object'].get('content'):
return
if 'www.youtube.com' not in postJsonObject['object']['content']:
contentStr = getBaseContentFromPost(postJsonObject, systemLanguage)
if 'www.youtube.com' not in contentStr:
return
postJsonObject['object']['content'] = \
postJsonObject['object']['content'].replace('www.youtube.com',
replacementDomain)
contentStr = contentStr.replace('www.youtube.com', replacementDomain)
postJsonObject['object']['content'] = contentStr
if postJsonObject['object'].get('contentMap'):
postJsonObject['object']['contentMap'][systemLanguage] = contentStr
def _removeMetaData(imageFilename: str, outputFilename: str) -> None:
@ -91,25 +96,64 @@ def _spoofMetaData(baseDir: str, nickname: str, domain: str,
camMake, camModel, camSerialNumber) = \
spoofGeolocation(baseDir, spoofCity, currTimeAdjusted,
decoySeed, None, None)
os.system('exiftool -artist="' + nickname + '" ' +
'-Make="' + camMake + '" ' +
'-Model="' + camModel + '" ' +
'-Comment="' + str(camSerialNumber) + '" ' +
'-DateTimeOriginal="' + published + '" ' +
'-FileModifyDate="' + published + '" ' +
'-CreateDate="' + published + '" ' +
'-GPSLongitudeRef=' + longitudeRef + ' ' +
'-GPSAltitude=0 ' +
'-GPSLongitude=' + str(longitude) + ' ' +
'-GPSLatitudeRef=' + latitudeRef + ' ' +
'-GPSLatitude=' + str(latitude) + ' ' +
'-Comment="" ' +
outputFilename) # nosec
if os.system('exiftool -artist="' + nickname + '" ' +
'-Make="' + camMake + '" ' +
'-Model="' + camModel + '" ' +
'-Comment="' + str(camSerialNumber) + '" ' +
'-DateTimeOriginal="' + published + '" ' +
'-FileModifyDate="' + published + '" ' +
'-CreateDate="' + published + '" ' +
'-GPSLongitudeRef=' + longitudeRef + ' ' +
'-GPSAltitude=0 ' +
'-GPSLongitude=' + str(longitude) + ' ' +
'-GPSLatitudeRef=' + latitudeRef + ' ' +
'-GPSLatitude=' + str(latitude) + ' ' +
'-Comment="" ' +
outputFilename) != 0: # nosec
print('ERROR: exiftool failed to run')
else:
print('ERROR: exiftool is not installed')
return
def convertImageToLowBandwidth(imageFilename: str) -> None:
"""Converts an image to a low bandwidth version
"""
lowBandwidthFilename = imageFilename + '.low'
if os.path.isfile(lowBandwidthFilename):
try:
os.remove(lowBandwidthFilename)
except BaseException:
pass
cmd = \
'/usr/bin/convert +noise Multiplicative ' + \
'-evaluate median 10% -dither Floyd-Steinberg ' + \
'-monochrome ' + imageFilename + ' ' + lowBandwidthFilename
print('Low bandwidth image conversion: ' + cmd)
subprocess.call(cmd, shell=True)
# wait for conversion to happen
ctr = 0
while not os.path.isfile(lowBandwidthFilename):
print('Waiting for low bandwidth image conversion ' + str(ctr))
time.sleep(0.2)
ctr += 1
if ctr > 100:
print('WARN: timed out waiting for low bandwidth image conversion')
break
if os.path.isfile(lowBandwidthFilename):
try:
os.remove(imageFilename)
except BaseException:
pass
os.rename(lowBandwidthFilename, imageFilename)
if os.path.isfile(imageFilename):
print('Image converted to low bandwidth ' + imageFilename)
else:
print('Low bandwidth converted image not found: ' +
lowBandwidthFilename)
def processMetaData(baseDir: str, nickname: str, domain: str,
imageFilename: str, outputFilename: str,
city: str) -> None:
@ -205,7 +249,7 @@ def attachMedia(baseDir: str, httpPrefix: str,
nickname: str, domain: str, port: int,
postJson: {}, imageFilename: str,
mediaType: str, description: str,
city: str) -> {}:
city: str, lowBandwidth: bool) -> {}:
"""Attaches media to a json object post
The description can be None
"""
@ -258,6 +302,8 @@ def attachMedia(baseDir: str, httpPrefix: str,
if baseDir:
if mediaType.startswith('image/'):
if lowBandwidth:
convertImageToLowBandwidth(imageFilename)
processMetaData(baseDir, nickname, domain,
imageFilename, mediaFilename, city)
else:

View File

@ -12,6 +12,7 @@ from utils import isAccountDir
from utils import getNicknameFromActor
from utils import getDomainFromActor
from utils import acctDir
from utils import hasGroupType
from webfinger import webfingerHandle
from blocking import isBlocked
from posts import getUserUrl
@ -58,7 +59,7 @@ def _updateMovedHandle(baseDir: str, nickname: str, domain: str,
handle = handle[1:]
wfRequest = webfingerHandle(session, handle,
httpPrefix, cachedWebfingers,
None, __version__, debug)
None, __version__, debug, False)
if not wfRequest:
print('updateMovedHandle unable to webfinger ' + handle)
return ctr
@ -102,13 +103,14 @@ def _updateMovedHandle(baseDir: str, nickname: str, domain: str,
if movedToPort:
if movedToPort != 80 and movedToPort != 443:
movedToDomainFull = movedToDomain + ':' + str(movedToPort)
groupAccount = hasGroupType(baseDir, movedToUrl, None)
if isBlocked(baseDir, nickname, domain,
movedToNickname, movedToDomain):
# someone that you follow has moved to a blocked domain
# so just unfollow them
unfollowAccount(baseDir, nickname, domain,
movedToNickname, movedToDomainFull,
'following.txt', debug)
debug, groupAccount, 'following.txt')
return ctr
followingFilename = acctDir(baseDir, nickname, domain) + '/following.txt'
@ -134,7 +136,7 @@ def _updateMovedHandle(baseDir: str, nickname: str, domain: str,
unfollowAccount(baseDir, nickname, domain,
handleNickname,
handleDomain,
'following.txt', debug)
debug, groupAccount, 'following.txt')
ctr += 1
print('Unfollowed ' + handle + ' who has moved to ' +
movedToHandle)

View File

@ -25,6 +25,7 @@ from newswire import getDictFromNewswire
from posts import createNewsPost
from posts import archivePostsForPerson
from content import validHashTag
from utils import getBaseContentFromPost
from utils import removeHtml
from utils import getFullDomain
from utils import loadJson
@ -32,6 +33,7 @@ from utils import saveJson
from utils import getStatusNumber
from utils import clearFromPostCaches
from utils import dangerousMarkup
from utils import localActorUrl
from inbox import storeHashTags
from session import createSession
@ -279,7 +281,7 @@ def hashtagRuleTree(operators: [],
def _hashtagAdd(baseDir: str, httpPrefix: str, domainFull: str,
postJsonObject: {},
actionStr: str, hashtags: []) -> None:
actionStr: str, hashtags: [], systemLanguage: str) -> None:
"""Adds a hashtag via a hashtag rule
"""
addHashtag = actionStr.split('add ', 1)[1].strip()
@ -313,7 +315,7 @@ def _hashtagAdd(baseDir: str, httpPrefix: str, domainFull: str,
hashtagHtml = \
" <a href=\"" + hashtagUrl + "\" class=\"addedHashtag\" " + \
"rel=\"tag\">#<span>" + htId + "</span></a>"
content = postJsonObject['object']['content']
content = getBaseContentFromPost(postJsonObject, systemLanguage)
if hashtagHtml in content:
return
@ -328,7 +330,7 @@ def _hashtagAdd(baseDir: str, httpPrefix: str, domainFull: str,
def _hashtagRemove(httpPrefix: str, domainFull: str, postJsonObject: {},
actionStr: str, hashtags: []) -> None:
actionStr: str, hashtags: [], systemLanguage: str) -> None:
"""Removes a hashtag via a hashtag rule
"""
rmHashtag = actionStr.split('remove ', 1)[1].strip()
@ -343,10 +345,11 @@ def _hashtagRemove(httpPrefix: str, domainFull: str, postJsonObject: {},
hashtagHtml = \
"<a href=\"" + hashtagUrl + "\" class=\"addedHashtag\" " + \
"rel=\"tag\">#<span>" + htId + "</span></a>"
content = postJsonObject['object']['content']
content = getBaseContentFromPost(postJsonObject, systemLanguage)
if hashtagHtml in content:
content = content.replace(hashtagHtml, '').replace(' ', ' ')
postJsonObject['object']['content'] = content
postJsonObject['object']['contentMap'][systemLanguage] = content
rmTagObject = None
for t in postJsonObject['object']['tag']:
if t.get('type') and t.get('name'):
@ -365,7 +368,8 @@ def _newswireHashtagProcessing(session, baseDir: str, postJsonObject: {},
cachedWebfingers: {},
federationList: [],
sendThreads: [], postLog: [],
moderated: bool, url: str) -> bool:
moderated: bool, url: str,
systemLanguage: str) -> bool:
"""Applies hashtag rules to a news post.
Returns true if the post should be saved to the news timeline
of this instance
@ -382,7 +386,7 @@ def _newswireHashtagProcessing(session, baseDir: str, postJsonObject: {},
# get the full text content of the post
content = ''
if postJsonObject['object'].get('content'):
content += postJsonObject['object']['content']
content += getBaseContentFromPost(postJsonObject, systemLanguage)
if postJsonObject['object'].get('summary'):
content += ' ' + postJsonObject['object']['summary']
content = content.lower()
@ -409,11 +413,11 @@ def _newswireHashtagProcessing(session, baseDir: str, postJsonObject: {},
if actionStr.startswith('add '):
# add a hashtag
_hashtagAdd(baseDir, httpPrefix, domainFull,
postJsonObject, actionStr, hashtags)
postJsonObject, actionStr, hashtags, systemLanguage)
elif actionStr.startswith('remove '):
# remove a hashtag
_hashtagRemove(httpPrefix, domainFull, postJsonObject,
actionStr, hashtags)
actionStr, hashtags, systemLanguage)
elif actionStr.startswith('block') or actionStr.startswith('drop'):
# Block this item
return False
@ -516,7 +520,9 @@ def _convertRSStoActivityPub(baseDir: str, httpPrefix: str,
federationList: [],
sendThreads: [], postLog: [],
maxMirroredArticles: int,
allowLocalNetworkAccess: bool) -> None:
allowLocalNetworkAccess: bool,
systemLanguage: str,
lowBandwidth: bool) -> None:
"""Converts rss items in a newswire into posts
"""
if not newswire:
@ -542,8 +548,8 @@ def _convertRSStoActivityPub(baseDir: str, httpPrefix: str,
statusNumber, published = getStatusNumber(dateStr)
newPostId = \
httpPrefix + '://' + domain + \
'/users/news/statuses/' + statusNumber
localActorUrl(httpPrefix, 'news', domain) + \
'/statuses/' + statusNumber
# file where the post is stored
filename = basePath + '/' + newPostId.replace('/', '#') + '.json'
@ -590,13 +596,15 @@ def _convertRSStoActivityPub(baseDir: str, httpPrefix: str,
mediaType = None
imageDescription = None
city = 'London, England'
conversationId = None
blog = createNewsPost(baseDir,
domain, port, httpPrefix,
rssDescription,
followersOnly, saveToFile,
attachImageFilename, mediaType,
imageDescription, city,
rssTitle)
rssTitle, systemLanguage,
conversationId, lowBandwidth)
if not blog:
continue
@ -606,7 +614,7 @@ def _convertRSStoActivityPub(baseDir: str, httpPrefix: str,
continue
idStr = \
httpPrefix + '://' + domain + '/users/news' + \
localActorUrl(httpPrefix, 'news', domain) + \
'/statuses/' + statusNumber + '/replies'
blog['news'] = True
@ -626,7 +634,7 @@ def _convertRSStoActivityPub(baseDir: str, httpPrefix: str,
blog['object']['published'] = dateStr
blog['object']['content'] = rssDescription
blog['object']['contentMap']['en'] = rssDescription
blog['object']['contentMap'][systemLanguage] = rssDescription
domainFull = getFullDomain(domain, port)
@ -641,7 +649,7 @@ def _convertRSStoActivityPub(baseDir: str, httpPrefix: str,
personCache, cachedWebfingers,
federationList,
sendThreads, postLog,
moderated, url)
moderated, url, systemLanguage)
# save the post and update the index
if savePost:
@ -663,7 +671,7 @@ def _convertRSStoActivityPub(baseDir: str, httpPrefix: str,
"\" class=\"addedHashtag\" " + \
"rel=\"tag\">#<span>" + \
htId + "</span></a>"
content = blog['object']['content']
content = getBaseContentFromPost(blog, systemLanguage)
if hashtagHtml not in content:
if content.endswith('</p>'):
content = \
@ -672,6 +680,7 @@ def _convertRSStoActivityPub(baseDir: str, httpPrefix: str,
else:
content += hashtagHtml
blog['object']['content'] = content
blog['object']['contentMap'][systemLanguage] = content
# update the newswire tags if new ones have been found by
# _newswireHashtagProcessing
@ -748,7 +757,8 @@ def runNewswireDaemon(baseDir: str, httpd,
httpd.maxTags,
httpd.maxFeedItemSizeKb,
httpd.maxNewswirePosts,
httpd.maxCategoriesFeedItemSizeKb)
httpd.maxCategoriesFeedItemSizeKb,
httpd.systemLanguage)
if not httpd.newswire:
if os.path.isfile(newswireStateFilename):
@ -773,7 +783,9 @@ def runNewswireDaemon(baseDir: str, httpd,
httpd.sendThreads,
httpd.postLog,
httpd.maxMirroredArticles,
httpd.allowLocalNetworkAccess)
httpd.allowLocalNetworkAccess,
httpd.systemLanguage,
httpd.lowBandwidth)
print('Newswire feed converted to ActivityPub')
if httpd.maxNewsPosts > 0:

View File

@ -18,6 +18,7 @@ from datetime import timezone
from collections import OrderedDict
from utils import validPostDate
from categories import setHashtagCategory
from utils import getBaseContentFromPost
from utils import hasObjectDict
from utils import firstParagraphFromString
from utils import isPublicPost
@ -29,6 +30,7 @@ from utils import containsInvalidChars
from utils import removeHtml
from utils import isAccountDir
from utils import acctDir
from utils import localActorUrl
from blocking import isBlockedDomain
from blocking import isBlockedHashtag
from filters import isFiltered
@ -67,8 +69,9 @@ def rss2Header(httpPrefix: str,
else:
rssStr += \
' <title>' + translate[title] + '</title>' + \
' <link>' + httpPrefix + '://' + domainFull + \
'/users/' + nickname + '/rss.xml' + '</link>'
' <link>' + \
localActorUrl(httpPrefix, nickname, domainFull) + \
'/rss.xml' + '</link>'
return rssStr
@ -290,7 +293,8 @@ def _xml2StrToHashtagCategories(baseDir: str, xmlStr: str,
hashtagList = hashtagListStr.split(' ')
if not isBlockedHashtag(baseDir, categoryStr):
for hashtag in hashtagList:
setHashtagCategory(baseDir, hashtag, categoryStr, force)
setHashtagCategory(baseDir, hashtag, categoryStr,
False, force)
def _xml2StrToDict(baseDir: str, domain: str, xmlStr: str,
@ -909,7 +913,7 @@ def _addAccountBlogsToNewswire(baseDir: str, nickname: str, domain: str,
newswire: {},
maxBlogsPerAccount: int,
indexFilename: str,
maxTags: int) -> None:
maxTags: int, systemLanguage: str) -> None:
"""Adds blogs for the given account to the newswire
"""
if not os.path.isfile(indexFilename):
@ -961,7 +965,8 @@ def _addAccountBlogsToNewswire(baseDir: str, nickname: str, domain: str,
votes = []
if os.path.isfile(fullPostFilename + '.votes'):
votes = loadJson(fullPostFilename + '.votes')
content = postJsonObject['object']['content']
content = \
getBaseContentFromPost(postJsonObject, systemLanguage)
description = firstParagraphFromString(content)
description = removeHtml(description)
tagsFromPost = _getHashtagsFromPost(postJsonObject)
@ -981,7 +986,7 @@ def _addAccountBlogsToNewswire(baseDir: str, nickname: str, domain: str,
def _addBlogsToNewswire(baseDir: str, domain: str, newswire: {},
maxBlogsPerAccount: int,
maxTags: int) -> None:
maxTags: int, systemLanguage: str) -> None:
"""Adds blogs from each user account into the newswire
"""
moderationDict = {}
@ -1009,7 +1014,8 @@ def _addBlogsToNewswire(baseDir: str, domain: str, newswire: {},
domain = handle.split('@')[1]
_addAccountBlogsToNewswire(baseDir, nickname, domain,
newswire, maxBlogsPerAccount,
blogsIndex, maxTags)
blogsIndex, maxTags,
systemLanguage)
break
# sort the moderation dict into chronological order, latest first
@ -1029,7 +1035,8 @@ def getDictFromNewswire(session, baseDir: str, domain: str,
maxPostsPerSource: int, maxFeedSizeKb: int,
maxTags: int, maxFeedItemSizeKb: int,
maxNewswirePosts: int,
maxCategoriesFeedItemSizeKb: int) -> {}:
maxCategoriesFeedItemSizeKb: int,
systemLanguage: str) -> {}:
"""Gets rss feeds as a dictionary from newswire file
"""
subscriptionsFilename = baseDir + '/accounts/newswire.txt'
@ -1077,7 +1084,7 @@ def getDictFromNewswire(session, baseDir: str, domain: str,
# add blogs from each user account
_addBlogsToNewswire(baseDir, domain, result,
maxPostsPerSource, maxTags)
maxPostsPerSource, maxTags, systemLanguage)
# sort into chronological order, latest first
sortedResult = OrderedDict(sorted(result.items(), reverse=True))

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,28 @@
{
"@context":{
"dfc-p": "http://static.datafoodconsortium.org/ontologies/dfc_ProductGlossary.owl#",
"dfc-u":"http://static.datafoodconsortium.org/data/units.rdf#"
},
"@graph":[
{
"@id":"dfc-u:kg",
"@type":"dfc-p:Unit",
"rdfs:label":"kilogramme"
},
{
"@id":"dfc-u:u",
"@type":"dfc-p:Unit",
"rdfs:label":"unité"
},
{
"@id":"dfc-u:g",
"@type":"dfc-p:Unit",
"rdfs:label":"gramme"
},
{
"@id":"dfc-u:l",
"@type":"dfc-p:Unit",
"rdfs:label":"litre"
}
]
}

View File

@ -16,6 +16,7 @@ from posts import outboxMessageCreateWrap
from posts import savePostToBox
from posts import sendToFollowersThread
from posts import sendToNamedAddresses
from utils import getBaseContentFromPost
from utils import hasObjectDict
from utils import getLocalNetworkAddresses
from utils import getFullDomain
@ -26,6 +27,7 @@ from utils import isFeaturedWriter
from utils import loadJson
from utils import saveJson
from utils import acctDir
from utils import localActorUrl
from blocking import isBlockedDomain
from blocking import outboxBlock
from blocking import outboxUndoBlock
@ -61,7 +63,10 @@ def _outboxPersonReceiveUpdate(recentPostsCache: {},
if not messageJson.get('type'):
return
print("messageJson['type'] " + messageJson['type'])
if not isinstance(messageJson['type'], str):
if debug:
print('DEBUG: c2s actor update type is not a string')
return
if messageJson['type'] != 'Update':
return
if not hasObjectDict(messageJson):
@ -72,6 +77,10 @@ def _outboxPersonReceiveUpdate(recentPostsCache: {},
if debug:
print('DEBUG: c2s actor update - no type')
return
if not isinstance(messageJson['object']['type'], str):
if debug:
print('DEBUG: c2s actor update object type is not a string')
return
if messageJson['object']['type'] != 'Person':
if debug:
print('DEBUG: not a c2s actor update')
@ -88,17 +97,21 @@ def _outboxPersonReceiveUpdate(recentPostsCache: {},
if debug:
print('DEBUG: c2s actor update has no id field')
return
actor = \
httpPrefix + '://' + getFullDomain(domain, port) + '/users/' + nickname
if not isinstance(messageJson['id'], str):
if debug:
print('DEBUG: c2s actor update id is not a string')
return
domainFull = getFullDomain(domain, port)
actor = localActorUrl(httpPrefix, nickname, domainFull)
if len(messageJson['to']) != 1:
if debug:
print('DEBUG: c2s actor update - to does not contain one actor ' +
messageJson['to'])
str(messageJson['to']))
return
if messageJson['to'][0] != actor:
if debug:
print('DEBUG: c2s actor update - to does not contain actor ' +
messageJson['to'] + ' ' + actor)
str(messageJson['to']) + ' ' + actor)
return
if not messageJson['id'].startswith(actor + '#updates/'):
if debug:
@ -178,7 +191,10 @@ def postMessageToOutbox(session, translate: {},
YTReplacementDomain: str,
showPublishedDateOnly: bool,
allowLocalNetworkAccess: bool,
city: str) -> bool:
city: str, systemLanguage: str,
sharedItemsFederatedDomains: [],
sharedItemFederationTokens: {},
lowBandwidth: bool) -> bool:
"""post is received by the outbox
Client to server message post
https://www.w3.org/TR/activitypub/#client-to-server-outbox-delivery
@ -201,9 +217,9 @@ def postMessageToOutbox(session, translate: {},
# check that the outgoing post doesn't contain any markup
# which can be used to implement exploits
if hasObjectDict(messageJson):
if messageJson['object'].get('content'):
if dangerousMarkup(messageJson['object']['content'],
allowLocalNetworkAccess):
contentStr = getBaseContentFromPost(messageJson, systemLanguage)
if contentStr:
if dangerousMarkup(contentStr, allowLocalNetworkAccess):
print('POST to outbox contains dangerous markup: ' +
str(messageJson))
return False
@ -264,7 +280,7 @@ def postMessageToOutbox(session, translate: {},
print('DEBUG: domain is blocked: ' + messageJson['actor'])
return False
# replace youtube, so that google gets less tracking data
replaceYouTube(messageJson, YTReplacementDomain)
replaceYouTube(messageJson, YTReplacementDomain, systemLanguage)
# https://www.w3.org/TR/activitypub/#create-activity-outbox
messageJson['object']['attributedTo'] = messageJson['actor']
if messageJson['object'].get('attachment'):
@ -378,7 +394,7 @@ def postMessageToOutbox(session, translate: {},
if messageJson['type'] in indexedActivities:
indexes = [outboxName, "inbox"]
selfActor = \
httpPrefix + '://' + domainFull + '/users/' + postToNickname
localActorUrl(httpPrefix, postToNickname, domainFull)
for boxNameIndex in indexes:
if not boxNameIndex:
continue
@ -390,7 +406,8 @@ def postMessageToOutbox(session, translate: {},
messageJson,
translate, YTReplacementDomain,
allowLocalNetworkAccess,
recentPostsCache, debug):
recentPostsCache, debug, systemLanguage,
domainFull, personCache):
inboxUpdateIndex('tlmedia', baseDir,
postToNickname + '@' + domain,
savedFilename, debug)
@ -449,7 +466,9 @@ def postMessageToOutbox(session, translate: {},
cachedWebfingers,
personCache,
messageJson, debug,
version)
version,
sharedItemsFederatedDomains,
sharedItemFederationTokens)
followersThreads.append(followersThread)
if debug:
@ -535,9 +554,9 @@ def postMessageToOutbox(session, translate: {},
if debug:
print('DEBUG: handle share uploads')
outboxShareUpload(baseDir, httpPrefix,
postToNickname, domain,
port, messageJson, debug, city)
outboxShareUpload(baseDir, httpPrefix, postToNickname, domain,
port, messageJson, debug, city,
systemLanguage, translate, lowBandwidth)
if debug:
print('DEBUG: handle undo share uploads')
@ -571,5 +590,7 @@ def postMessageToOutbox(session, translate: {},
cachedWebfingers,
personCache,
messageJson, debug,
version)
version,
sharedItemsFederatedDomains,
sharedItemFederationTokens)
return True

View File

@ -37,6 +37,7 @@ from roles import setRole
from roles import setRolesFromList
from roles import getActorRolesList
from media import processMetaData
from utils import removeLineEndings
from utils import removeDomainPort
from utils import getStatusNumber
from utils import getFullDomain
@ -50,8 +51,10 @@ from utils import getProtocolPrefixes
from utils import hasUsersPath
from utils import getImageExtensions
from utils import isImageFile
from utils import getUserPaths
from utils import acctDir
from utils import getUserPaths
from utils import getGroupPaths
from utils import localActorUrl
from session import createSession
from session import getJson
from webfinger import webfingerHandle
@ -136,8 +139,8 @@ def setProfileImage(baseDir: str, httpPrefix: str, nickname: str, domain: str,
if personJson:
personJson[iconFilenameBase]['mediaType'] = mediaType
personJson[iconFilenameBase]['url'] = \
httpPrefix + '://' + fullDomain + '/users/' + \
nickname + '/' + iconFilename
localActorUrl(httpPrefix, nickname, fullDomain) + \
'/' + iconFilename
saveJson(personJson, personFilename)
cmd = \
@ -226,13 +229,15 @@ def getDefaultPersonContext() -> str:
def _createPersonBase(baseDir: str, nickname: str, domain: str, port: int,
httpPrefix: str, saveToFile: bool,
manualFollowerApproval: bool,
password: str = None) -> (str, str, {}, {}):
groupAccount: bool,
password: str) -> (str, str, {}, {}):
"""Returns the private key, public key, actor and webfinger endpoint
"""
privateKeyPem, publicKeyPem = generateRSAKey()
webfingerEndpoint = \
createWebfingerEndpoint(nickname, domain, port,
httpPrefix, publicKeyPem)
httpPrefix, publicKeyPem,
groupAccount)
if saveToFile:
storeWebfingerEndpoint(nickname, domain, port,
baseDir, webfingerEndpoint)
@ -242,10 +247,12 @@ def _createPersonBase(baseDir: str, nickname: str, domain: str, port: int,
domain = getFullDomain(domain, port)
personType = 'Person'
if groupAccount:
personType = 'Group'
# Enable follower approval by default
approveFollowers = manualFollowerApproval
personName = nickname
personId = httpPrefix + '://' + domain + '/users/' + nickname
personId = localActorUrl(httpPrefix, nickname, domain)
inboxStr = personId + '/inbox'
personUrl = httpPrefix + '://' + domain + '/@' + personName
if nickname == 'inbox':
@ -294,7 +301,7 @@ def _createPersonBase(baseDir: str, nickname: str, domain: str, port: int,
'followers': personId + '/followers',
'following': personId + '/following',
'tts': personId + '/speaker',
'shares': personId + '/shares',
'shares': personId + '/catalog',
'hasOccupation': [
{
'@type': 'Occupation',
@ -396,6 +403,7 @@ def _createPersonBase(baseDir: str, nickname: str, domain: str, port: int,
print(publicKeyPem, file=text_file)
if password:
password = removeLineEndings(password)
storeBasicCredentials(baseDir, nickname, password)
return privateKeyPem, publicKeyPem, newPerson, webfingerEndpoint
@ -434,8 +442,8 @@ def createGroup(baseDir: str, nickname: str, domain: str, port: int,
newPerson, webfingerEndpoint) = createPerson(baseDir, nickname,
domain, port,
httpPrefix, saveToFile,
False, password)
newPerson['type'] = 'Group'
False, password, True)
return privateKeyPem, publicKeyPem, newPerson, webfingerEndpoint
@ -456,7 +464,8 @@ def savePersonQrcode(baseDir: str,
def createPerson(baseDir: str, nickname: str, domain: str, port: int,
httpPrefix: str, saveToFile: bool,
manualFollowerApproval: bool,
password: str = None) -> (str, str, {}, {}):
password: str,
groupAccount: bool = False) -> (str, str, {}, {}):
"""Returns the private key, public key, actor and webfinger endpoint
"""
if not validNickname(domain, nickname):
@ -482,6 +491,7 @@ def createPerson(baseDir: str, nickname: str, domain: str, port: int,
httpPrefix,
saveToFile,
manualFollowerApproval,
groupAccount,
password)
if not getConfigParam(baseDir, 'admin'):
if nickname != 'news':
@ -552,7 +562,7 @@ def createSharedInbox(baseDir: str, nickname: str, domain: str, port: int,
"""Generates the shared inbox
"""
return _createPersonBase(baseDir, nickname, domain, port, httpPrefix,
True, True, None)
True, True, False, None)
def createNewsInbox(baseDir: str, domain: str, port: int,
@ -584,6 +594,11 @@ def personUpgradeActor(baseDir: str, personJson: {},
personJson['published'] = published
updateActor = True
if personJson.get('shares'):
if personJson['shares'].endswith('/shares'):
personJson['shares'] = personJson['id'] + '/catalog'
updateActor = True
occupationName = ''
if personJson.get('occupationName'):
occupationName = personJson['occupationName']
@ -746,6 +761,7 @@ def personBoxJson(recentPostsCache: {},
boxname != 'tlfeatures' and \
boxname != 'outbox' and boxname != 'moderation' and \
boxname != 'tlbookmarks' and boxname != 'bookmarks':
print('ERROR: personBoxJson invalid box name ' + boxname)
return None
if not '/' + boxname in path:
@ -1186,6 +1202,18 @@ def setPersonNotes(baseDir: str, nickname: str, domain: str,
return True
def _detectUsersPath(url: str) -> str:
"""Tries to detect the /users/ path
"""
if '/' not in url:
return '/users/'
usersPaths = getUserPaths()
for possibleUsersPath in usersPaths:
if possibleUsersPath in url:
return possibleUsersPath
return '/users/'
def getActorJson(hostDomain: str, handle: str, http: bool, gnunet: bool,
debug: bool, quiet: bool = False) -> ({}, {}):
"""Returns the actor json
@ -1193,21 +1221,29 @@ def getActorJson(hostDomain: str, handle: str, http: bool, gnunet: bool,
if debug:
print('getActorJson for ' + handle)
originalActor = handle
groupAccount = False
# try to determine the users path
detectedUsersPath = _detectUsersPath(handle)
if '/@' in handle or \
'/users/' in handle or \
detectedUsersPath in handle or \
handle.startswith('http') or \
handle.startswith('hyper'):
groupPaths = getGroupPaths()
if detectedUsersPath in groupPaths:
groupAccount = True
# format: https://domain/@nick
originalHandle = handle
if not hasUsersPath(originalHandle):
if not quiet or debug:
print('getActorJson: Expected actor format: ' +
'https://domain/@nick or https://domain/users/nick')
'https://domain/@nick or https://domain' +
detectedUsersPath + 'nick')
return None, None
prefixes = getProtocolPrefixes()
for prefix in prefixes:
handle = handle.replace(prefix, '')
handle = handle.replace('/@', '/users/')
handle = handle.replace('/@', detectedUsersPath)
paths = getUserPaths()
userPathFound = False
for userPath in paths:
@ -1234,6 +1270,10 @@ def getActorJson(hostDomain: str, handle: str, http: bool, gnunet: bool,
return None, None
if handle.startswith('@'):
handle = handle[1:]
elif handle.startswith('!'):
# handle for a group
handle = handle[1:]
groupAccount = True
if '@' not in handle:
if not quiet:
print('getActorJsonSyntax: --actor nickname@domain')
@ -1265,7 +1305,8 @@ def getActorJson(hostDomain: str, handle: str, http: bool, gnunet: bool,
handle = nickname + '@' + domain
wfRequest = webfingerHandle(session, handle,
httpPrefix, cachedWebfingers,
None, __version__, debug)
None, __version__, debug,
groupAccount)
if not wfRequest:
if not quiet:
print('getActorJson Unable to webfinger ' + handle)
@ -1282,7 +1323,8 @@ def getActorJson(hostDomain: str, handle: str, http: bool, gnunet: bool,
personUrl = None
if wfRequest.get('errors'):
if not quiet or debug:
print('getActorJson wfRequest error: ' + str(wfRequest['errors']))
print('getActorJson wfRequest error: ' +
str(wfRequest['errors']))
if hasUsersPath(handle):
personUrl = originalActor
else:

5
pgp.py
View File

@ -15,6 +15,7 @@ from utils import containsPGPPublicKey
from utils import isPGPEncrypted
from utils import getFullDomain
from utils import getStatusNumber
from utils import localActorUrl
from webfinger import webfingerHandle
from posts import getPersonBox
from auth import createBasicAuthHeader
@ -489,7 +490,7 @@ def pgpPublicKeyUpload(baseDir: str, session,
if debug:
print('Actor for ' + handle + ' obtained')
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
handle = actor.replace('/users/', '/@')
# check that this looks like the correct actor
@ -547,7 +548,7 @@ def pgpPublicKeyUpload(baseDir: str, session,
# lookup the inbox for the To handle
wfRequest = \
webfingerHandle(session, handle, httpPrefix, cachedWebfingers,
domain, __version__, debug)
domain, __version__, debug, False)
if not wfRequest:
if debug:
print('DEBUG: pgp actor update webfinger failed for ' +

832
posts.py

File diff suppressed because it is too large Load Diff

View File

@ -39,6 +39,7 @@ from collections import deque, namedtuple
from numbers import Integral, Real
from context import getApschemaV1_9
from context import getApschemaV1_20
from context import getApschemaV1_21
from context import getLitepubV0_1
from context import getLitepubSocial
@ -408,6 +409,13 @@ def load_document(url):
'document': getApschemaV1_9()
}
return doc
elif url.endswith('/apschema/v1.20'):
doc = {
'contextUrl': None,
'documentUrl': url,
'document': getApschemaV1_20()
}
return doc
elif url.endswith('/apschema/v1.21'):
doc = {
'contextUrl': None,

View File

@ -112,7 +112,10 @@ def _updatePostSchedule(baseDir: str, handle: str, httpd,
httpd.YTReplacementDomain,
httpd.showPublishedDateOnly,
httpd.allowLocalNetworkAccess,
httpd.city):
httpd.city, httpd.systemLanguage,
httpd.sharedItemsFederatedDomains,
httpd.sharedItemFederationTokens,
httpd.lowBandwidth):
indexLines.remove(line)
os.remove(postFilename)
continue

View File

@ -279,6 +279,21 @@ function notifications {
fi
fi
# send notifications for new wanted items to XMPP/email users
epicyonWantedFile="$epicyonDir/.newWanted"
if [ -f "$epicyonWantedFile" ]; then
if ! grep -q "##sent##" "$epicyonWantedFile"; then
epicyonWantedMessage=$(notification_translate_text 'Wanted')
epicyonWantedFileContent=$(echo "$epicyonWantedMessage")" "$(cat "$epicyonWantedFile")
if [[ "$epicyonWantedFileContent" == *':'* ]]; then
epicyonWantedMessage="Epicyon: $epicyonWantedFileContent"
fi
sendNotification "$USERNAME" "Epicyon" "$epicyonWantedMessage"
echo "##sent##" > "$epicyonWantedFile"
chown ${PROJECT_NAME}:${PROJECT_NAME} "$epicyonWantedFile"
fi
fi
# send notifications for follow requests to XMPP/email users
epicyonFollowFile="$epicyonDir/followrequests.txt"
epicyonFollowNotificationsFile="$epicyonDir/follownotifications.txt"

View File

@ -124,7 +124,8 @@ def getJson(session, url: str, headers: {}, params: {}, debug: bool,
else:
print('WARN: getJson url: ' + url +
' failed with error code ' +
str(result.status_code))
str(result.status_code) +
' headers: ' + str(sessionHeaders))
return result.json()
except requests.exceptions.RequestException as e:
sessionHeaders2 = sessionHeaders.copy()

1373
shares.py

File diff suppressed because it is too large Load Diff

View File

@ -19,6 +19,7 @@ from utils import loadJson
from utils import getOccupationSkills
from utils import setOccupationSkillsList
from utils import acctDir
from utils import localActorUrl
def setSkillsFromDict(actorJson: {}, skillsDict: {}) -> []:
@ -185,7 +186,7 @@ def sendSkillViaServer(baseDir: str, session, nickname: str, password: str,
domainFull = getFullDomain(domain, port)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
toUrl = actor
ccUrl = actor + '/followers'
@ -208,7 +209,7 @@ def sendSkillViaServer(baseDir: str, session, nickname: str, password: str,
wfRequest = \
webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
domain, projectVersion, debug, False)
if not wfRequest:
if debug:
print('DEBUG: skill webfinger failed for ' + handle)

View File

@ -17,7 +17,8 @@ from utils import getFullDomain
def instancesGraph(baseDir: str, handles: str,
proxyType: str,
port: int, httpPrefix: str,
debug: bool, projectVersion: str) -> str:
debug: bool, projectVersion: str,
systemLanguage: str) -> str:
""" Returns a dot graph of federating instances
based upon a few sample handles.
The handles argument should contain a comma separated list
@ -53,7 +54,7 @@ def instancesGraph(baseDir: str, handles: str,
wfRequest = \
webfingerHandle(session, handle, httpPrefix,
cachedWebfingers,
domain, projectVersion, debug)
domain, projectVersion, debug, False)
if not wfRequest:
return dotGraphStr + '}\n'
if not isinstance(wfRequest, dict):
@ -74,7 +75,7 @@ def instancesGraph(baseDir: str, handles: str,
maxAttachments, federationList,
personCache, debug,
projectVersion, httpPrefix, domain,
wordFrequency, [])
wordFrequency, [], systemLanguage)
postDomains.sort()
for fedDomain in postDomains:
dotLineStr = ' "' + domain + '" -> "' + fedDomain + '";\n'

View File

@ -24,6 +24,7 @@ from utils import saveJson
from utils import isPGPEncrypted
from utils import hasObjectDict
from utils import acctDir
from utils import localActorUrl
from content import htmlReplaceQuoteMarks
speakerRemoveChars = ('.\n', '. ', ',', ';', '?', '!')
@ -452,7 +453,7 @@ def _postToSpeakerJson(baseDir: str, httpPrefix: str,
img['name'] + '. '
isDirect = isDM(postJsonObject)
actor = httpPrefix + '://' + domainFull + '/users/' + nickname
actor = localActorUrl(httpPrefix, nickname, domainFull)
replyToYou = isReply(postJsonObject, actor)
published = ''

1220
tests.py

File diff suppressed because it is too large Load Diff

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.1 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.4 KiB

After

Width:  |  Height:  |  Size: 6.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 8.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.7 KiB

After

Width:  |  Height:  |  Size: 9.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 198 B

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 992 B

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 KiB

After

Width:  |  Height:  |  Size: 7.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.0 KiB

After

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 KiB

After

Width:  |  Height:  |  Size: 8.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 KiB

After

Width:  |  Height:  |  Size: 6.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.1 KiB

View File

@ -14,7 +14,7 @@
"post-separator-margin-top": "10px",
"post-separator-margin-bottom": "10px",
"vertical-between-posts": "10px",
"time-vertical-align": "10px",
"time-vertical-align": "0%",
"button-corner-radius": "5px",
"timeline-border-radius": "5px",
"newswire-publish-icon": "True",

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.4 KiB

After

Width:  |  Height:  |  Size: 6.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.9 KiB

After

Width:  |  Height:  |  Size: 8.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 9.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 198 B

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 KiB

After

Width:  |  Height:  |  Size: 7.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.0 KiB

After

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 KiB

After

Width:  |  Height:  |  Size: 8.9 KiB

Some files were not shown because too many files have changed in this diff Show More