diff --git a/README_architecture.md b/README_architecture.md index 74c996d94..4dcb87bec 100644 --- a/README_architecture.md +++ b/README_architecture.md @@ -16,6 +16,10 @@ Although it can be single user, this is not strictly a single user system. The design of this system is opinionated, and to a large extent informed by years of past experience in the fediverse. There is no claim to neutrality of any sort. Automatic removal of hellthreads and other common griefing tactics is an example of this. +### Privacy Sensitive Defaults + +Follow approval should be required by default. This gives the user a chance to see who wants to follow them and make a decision. Also by default direct messages should not be permitted except with accounts that you are following. This helps to reduce spam and harrassment from random accounts in the wider fediverse. The aim is for the user to have a good experience by default, even if they have not yet built up any sort of block list. + ### Resisting Centralization Centralization is characterized by the typical fixation upon "scale" within the software industry. Systems which scale, in the way which is commonly understood, mean that a few individuals can control the social lives of many, and extract value from them in often cynical and manipulative ways. @@ -24,7 +28,7 @@ In general, methods have been preferred which do not vertically scale. This incl Being hostile towards the common notion of scaling means that this system will be of no interest to "big tech" and can't easily be used within extractive economic models without needing a substantial rewrite. This avoids the typical cooption strategies in which large companies eventually take over what was originally software developed by grassroots activists to address real community needs. -This system should however be able to scale rhizomatically with the deployment of many small instances federated together. Instead of scaling up, scale out. In a network of many small instances nobody has overall control and corporate capture is much more unlikely. Small instances also minimize the bureaucratic requirements for governance processes, which at medium to large scale eventually becomes tyrannical. +This system should however be able to scale rhizomatically with the deployment of many small instances federated together. Instead of scaling up, scale out. In a network of many small instances nobody has overall control and corporate capture is far less feasible. Small instances also minimize the bureaucratic requirements for governance processes, which at medium to large scale eventually becomes tyrannical. ### Roles @@ -32,11 +36,11 @@ The roles within an instance are comparable to the crew roles onboard a ship, wi ### No Javascript -This is so that the system can be accessed and used normally with javascript in the web browser turned off. If you want to have good security then this is useful, since lack of javascript greatly reduces the attack surface and constrains adversaries to a limited number of vectors. +This is so that the system can be accessed and used normally with javascript in the web browser turned off. If you want to have good security then this is useful, since lack of javascript greatly reduces the attack surface and constrains adversaries to a limited number of vectors. Not using javascript also makes this system usable in shell based browsers such as Lynx, or other less common browsers, which helps to avoid being locked in to a browser duopoly. ### Block Crawlers -Ordinarily web crawlers would not be a problem, but in the context of a social network even having crawlers index public posts can create ethical dilemmas in some circumstances. News instances may allow crawlers, but other types of instances should block them. +Ordinarily web crawlers would not be a problem, but in the context of a social network even having crawlers index public posts can create ethical dilemmas in some circumstances. News and blogging instances may allow crawlers, but other types of instances should block them. ### No Local or Federated Timelines @@ -64,7 +68,6 @@ Where Json linked data signatures are supported there should not be arbitrary sc In general avoid using web frameworks and instead use local modules which are prefixed with *webapp_*. Web frameworks are built for conventional software engineering by large companies who are designing for scale. They typically have database dependencies and contain a lot of hardcoded Google stuff or other things which will leak metadata or be incompatible with onion routing. Keeping up with web frameworks is a constant firefight. They also create a massive attack surface requiring constant vigilance. - ## High Level Architecture The main modules are *epicyon.py* and *daemon.py*. *epicyon.py* is the commandline interface and *daemon.py* is the http server. diff --git a/README_customizations.md b/README_customizations.md index a6f86ad7d..462ee1cdd 100644 --- a/README_customizations.md +++ b/README_customizations.md @@ -26,6 +26,20 @@ When a moderator report is created the message at the top of the screen can be c Extra emoji can be added to the *emoji* directory and you should then update the **emoji/emoji.json** file, which maps the name to the filename (without the .png extension). +Another way to import emoji is to create a text file where each line is the url of the emoji png file and the emoji name, separated by a comma. + +```bash +https://somesite/emoji1.png, :emojiname1: +https://somesite/emoji2.png, :emojiname2: +https://somesite/emoji3.png, :emojiname3: +``` + +Then this can be imported with: + +```bash +python3 epicyon.py --import-emoji [textfile] +``` + ## Themes -If you want to create a new theme then the functions for that are within *theme.py*. These functions take the CSS templates and modify them. You will need to edit *themesDropdown* within *webinterface.py* and add the appropriate translations for the theme name. Themes are selectable from the profile screen of the administrator. +If you want to create a new theme then copy the *default* directory within the *theme* directory, rename it to your new theme name, then you can edit the colors and fonts within *theme.json*, and change the icons and banners. Themes are selectable from the graphic design section of the profile screen of the administrator, or of any accounts having the *artist* role. diff --git a/blocking.py b/blocking.py index d3eefc4e4..4bce78539 100644 --- a/blocking.py +++ b/blocking.py @@ -1016,19 +1016,29 @@ def load_cw_lists(base_dir: str, verbose: bool) -> {}: def add_cw_from_lists(post_json_object: {}, cw_lists: {}, translate: {}, - lists_enabled: str) -> None: + lists_enabled: str, system_language: str) -> None: """Adds content warnings by matching the post content against domains or keywords """ if not lists_enabled: return if not post_json_object['object'].get('content'): - return + if not post_json_object['object'].get('contentMap'): + return cw_text = '' if post_json_object['object'].get('summary'): cw_text = post_json_object['object']['summary'] - content = post_json_object['object']['content'] + content = None + if post_json_object['object'].get('contentMap'): + if post_json_object['object']['contentMap'].get(system_language): + content = \ + post_json_object['object']['contentMap'][system_language] + if not content: + if post_json_object['object'].get('content'): + content = post_json_object['object']['content'] + if not content: + return for name, item in cw_lists.items(): if name not in lists_enabled: continue diff --git a/content.py b/content.py index f59112776..2adf09956 100644 --- a/content.py +++ b/content.py @@ -1463,7 +1463,8 @@ def content_diff(content: str, prev_content: str) -> str: def create_edits_html(edits_json: {}, post_json_object: {}, - translate: {}, timezone: str) -> str: + translate: {}, timezone: str, + system_language: str) -> str: """ Creates html showing historical edits made to a post """ if not edits_json: @@ -1471,20 +1472,42 @@ def create_edits_html(edits_json: {}, post_json_object: {}, if not has_object_dict(post_json_object): return '' if not post_json_object['object'].get('content'): - return '' + if not post_json_object['object'].get('contentMap'): + return '' edit_dates_list = [] for modified, item in edits_json.items(): edit_dates_list.append(modified) edit_dates_list.sort(reverse=True) edits_str = '' - content = remove_html(post_json_object['object']['content']) + content = None + if post_json_object['object'].get('contentMap'): + if post_json_object['object']['contentMap'].get(system_language): + content = \ + post_json_object['object']['contentMap'][system_language] + if not content: + if post_json_object['object'].get('content'): + content = post_json_object['object']['content'] + if not content: + return '' + content = remove_html(content) for modified in edit_dates_list: prev_json = edits_json[modified] if not has_object_dict(prev_json): continue + prev_content = None if not prev_json['object'].get('content'): + if not prev_json['object'].get('contentMap'): + continue + if prev_json['object'].get('contentMap'): + if prev_json['object']['contentMap'].get(system_language): + prev_content = \ + prev_json['object']['contentMap'][system_language] + if not prev_content: + if prev_json['object'].get('content'): + prev_content = prev_json['object']['content'] + if not prev_content: continue - prev_content = remove_html(prev_json['object']['content']) + prev_content = remove_html(prev_content) if content == prev_content: continue diff = content_diff(content, prev_content) diff --git a/daemon.py b/daemon.py index c9a4aa6f8..41610d720 100644 --- a/daemon.py +++ b/daemon.py @@ -16911,7 +16911,7 @@ class PubServer(BaseHTTPRequestHandler): self.server.domain_full, self.server.text_mode_banner, access_keys, - False) + False, self.server.system_language) if msg: msg = msg.encode('utf-8') msglen = len(msg) @@ -16952,12 +16952,17 @@ class PubServer(BaseHTTPRequestHandler): self.server.domain_full, self.server.text_mode_banner, access_keys, - True).encode('utf-8') - msglen = len(msg) - self._set_headers('text/calendar', - msglen, cookie, calling_domain, - False) - self._write(msg) + True, + self.server.system_language) + if msg: + msg = msg.encode('utf-8') + msglen = len(msg) + self._set_headers('text/calendar', + msglen, cookie, calling_domain, + False) + self._write(msg) + else: + self._404() fitness_performance(getreq_start_time, self.server.fitness, '_GET', 'icalendar shown', self.server.debug) @@ -18362,7 +18367,8 @@ class PubServer(BaseHTTPRequestHandler): self.server.http_prefix, curr_etag, self.server.recent_dav_etags, - self.server.domain_full) + self.server.domain_full, + self.server.system_language) elif endpoint_type == 'delete': response_str = \ dav_delete_response(self.server.base_dir, diff --git a/happening.py b/happening.py index 947a993f7..39bfd350c 100644 --- a/happening.py +++ b/happening.py @@ -228,7 +228,7 @@ def _event_text_match(content: str, text_match: str) -> bool: def get_todays_events(base_dir: str, nickname: str, domain: str, curr_year: int, curr_month_number: int, curr_day_of_month: int, - text_match: str) -> {}: + text_match: str, system_language: str) -> {}: """Retrieves calendar events for today Returns a dictionary of lists containing Event and Place activities """ @@ -268,8 +268,16 @@ def get_todays_events(base_dir: str, nickname: str, domain: str, continue if post_json_object.get('object'): - if post_json_object['object'].get('content'): - content = post_json_object['object']['content'] + content = None + if post_json_object['object'].get('contentMap'): + sys_lang = system_language + if post_json_object['object']['contentMap'].get(sys_lang): + content = \ + post_json_object['object']['contentMap'][sys_lang] + if not content: + if post_json_object['object'].get('content'): + content = post_json_object['object']['content'] + if content: if not _event_text_match(content, text_match): continue @@ -465,14 +473,14 @@ def get_todays_events_icalendar(base_dir: str, nickname: str, domain: str, year: int, month_number: int, day_number: int, person_cache: {}, http_prefix: str, - text_match: str) -> str: + text_match: str, system_language: str) -> str: """Returns today's events in icalendar format """ day_events = None events = \ get_todays_events(base_dir, nickname, domain, year, month_number, day_number, - text_match) + text_match, system_language) if events: if events.get(str(day_number)): day_events = events[str(day_number)] @@ -1010,7 +1018,7 @@ def dav_report_response(base_dir: str, nickname: str, domain: str, depth: int, xml_str: str, person_cache: {}, http_prefix: str, curr_etag: str, recent_dav_etags: {}, - domain_full: str) -> str: + domain_full: str, system_language: str) -> str: """Returns the response to caldav REPORT """ if ' int: def edited_post_filename(base_dir: str, nickname: str, domain: str, post_json_object: {}, debug: bool, - max_time_diff_seconds: int) -> (str, {}): + max_time_diff_seconds: int, + system_language: str) -> (str, {}): """Returns the filename of the edited post """ if not has_object_dict(post_json_object): @@ -5545,8 +5553,17 @@ def edited_post_filename(base_dir: str, nickname: str, domain: str, return '', None if debug: print(post_id + ' might be an edit of ' + lastpost_id) - if words_similarity(lastpost_json['object']['content'], - post_json_object['object']['content'], 10) < 70: + lastpost_content = lastpost_json['object']['content'] + if lastpost_json['object'].get('contentMap'): + if lastpost_json['object']['contentMap'].get(system_language): + lastpost_content = \ + lastpost_json['object']['contentMap'][system_language] + content = post_json_object['object']['content'] + if post_json_object['object'].get('contentMap'): + if post_json_object['object']['contentMap'].get(system_language): + content = \ + post_json_object['object']['contentMap'][system_language] + if words_similarity(lastpost_content, content, 10) < 70: return '', None print(post_id + ' is an edit of ' + lastpost_id) return lastpost_filename, lastpost_json diff --git a/tests.py b/tests.py index 9e8f9fdec..35d2cfaa6 100644 --- a/tests.py +++ b/tests.py @@ -6504,6 +6504,7 @@ def _test_word_similarity() -> None: def _test_add_cw_lists(base_dir: str) -> None: print('test_add_CW_from_lists') translate = {} + system_language = "en" cw_lists = load_cw_lists(base_dir, True) assert cw_lists @@ -6514,7 +6515,8 @@ def _test_add_cw_lists(base_dir: str) -> None: "content": "" } } - add_cw_from_lists(post_json_object, cw_lists, translate, 'Murdoch press') + add_cw_from_lists(post_json_object, cw_lists, translate, 'Murdoch press', + system_language) assert post_json_object['object']['sensitive'] is False assert post_json_object['object']['summary'] is None @@ -6522,10 +6524,13 @@ def _test_add_cw_lists(base_dir: str) -> None: "object": { "sensitive": False, "summary": None, - "content": "Blah blah news.co.uk blah blah" + "contentMap": { + "en": "Blah blah news.co.uk blah blah" + } } } - add_cw_from_lists(post_json_object, cw_lists, translate, 'Murdoch press') + add_cw_from_lists(post_json_object, cw_lists, translate, 'Murdoch press', + system_language) assert post_json_object['object']['sensitive'] is True assert post_json_object['object']['summary'] == "Murdoch Press" @@ -6536,7 +6541,8 @@ def _test_add_cw_lists(base_dir: str) -> None: "content": "Blah blah news.co.uk blah blah" } } - add_cw_from_lists(post_json_object, cw_lists, translate, 'Murdoch press') + add_cw_from_lists(post_json_object, cw_lists, translate, 'Murdoch press', + system_language) assert post_json_object['object']['sensitive'] is True assert post_json_object['object']['summary'] == \ "Murdoch Press / Existing CW" @@ -6945,6 +6951,7 @@ def _test_diff_content() -> None: '

' assert result == expected + system_language = "en" translate = { "SHOW EDITS": "SHOW EDITS" } @@ -6973,13 +6980,16 @@ def _test_diff_content() -> None: }, "2020-12-14T00:07:34Z": { "object": { - "content": content2, + "contentMap": { + "en": content2 + }, "published": "2020-12-14T00:07:34Z" } } } html_str = \ - create_edits_html(edits_json, post_json_object, translate, timezone) + create_edits_html(edits_json, post_json_object, translate, + timezone, system_language) assert html_str expected = \ '
SHOW EDITS' + \ diff --git a/video.py b/video.py index 1fa03a72e..ae797c01c 100644 --- a/video.py +++ b/video.py @@ -87,7 +87,11 @@ def convert_video_to_note(base_dir: str, nickname: str, domain: str, post_json_object['license']['name']): return None content += '

' + post_json_object['license']['name'] + '

' - content += post_json_object['content'] + post_content = post_json_object['content'] + if post_json_object.get('contentMap'): + if post_json_object['contentMap'].get(system_language): + post_content = post_json_object['contentMap'][system_language] + content += post_content conversation_id = remove_id_ending(post_json_object['id']) diff --git a/webapp_calendar.py b/webapp_calendar.py index 309f99263..0347bf570 100644 --- a/webapp_calendar.py +++ b/webapp_calendar.py @@ -265,7 +265,7 @@ def html_calendar(person_cache: {}, css_cache: {}, translate: {}, base_dir: str, path: str, http_prefix: str, domain_full: str, text_mode_banner: str, access_keys: {}, - icalendar: bool) -> str: + icalendar: bool, system_language: str) -> str: """Show the calendar for a person """ domain = remove_domain_port(domain_full) @@ -327,12 +327,13 @@ def html_calendar(person_cache: {}, css_cache: {}, translate: {}, day_number, person_cache, http_prefix, - text_match) + text_match, + system_language) day_events = None events = \ get_todays_events(base_dir, nickname, domain, year, month_number, day_number, - text_match) + text_match, system_language) if events: if events.get(str(day_number)): day_events = events[str(day_number)] diff --git a/webapp_post.py b/webapp_post.py index 448a73f85..afafe1e4e 100644 --- a/webapp_post.py +++ b/webapp_post.py @@ -92,7 +92,8 @@ from blocking import add_cw_from_lists from reaction import html_emoji_reactions -def _html_post_metadata_open_graph(domain: str, post_json_object: {}) -> str: +def _html_post_metadata_open_graph(domain: str, post_json_object: {}, + system_language: str) -> str: """Returns html OpenGraph metadata for a post """ metadata = \ @@ -122,7 +123,11 @@ def _html_post_metadata_open_graph(domain: str, post_json_object: {}) -> str: "\" property=\"og:published_time\" />\n" if not obj_json.get('attachment') or obj_json.get('sensitive'): if obj_json.get('content') and not obj_json.get('sensitive'): - description = remove_html(obj_json['content']) + obj_content = obj_json['content'] + if obj_json.get('contentMap'): + if obj_json['contentMap'].get(system_language): + obj_content = obj_json['contentMap'][system_language] + description = remove_html(obj_content) metadata += \ " \n" @@ -150,7 +155,11 @@ def _html_post_metadata_open_graph(domain: str, post_json_object: {}) -> str: description = 'Attached: 1 audio' if description: if obj_json.get('content') and not obj_json.get('sensitive'): - description += '\n\n' + remove_html(obj_json['content']) + obj_content = obj_json['content'] + if obj_json.get('contentMap'): + if obj_json['contentMap'].get(system_language): + obj_content = obj_json['contentMap'][system_language] + description += '\n\n' + remove_html(obj_content) metadata += \ " \n" @@ -1490,7 +1499,7 @@ def individual_post_as_html(signing_priv_key_pem: str, edits_json = load_json(edits_filename, 0, 1) if edits_json: edits_str = create_edits_html(edits_json, post_json_object, - translate, timezone) + translate, timezone, system_language) message_id_str = '' if message_id: @@ -1936,7 +1945,8 @@ def individual_post_as_html(signing_priv_key_pem: str, container_class = 'container dm' # add any content warning from the cwlists directory - add_cw_from_lists(post_json_object, cw_lists, translate, lists_enabled) + add_cw_from_lists(post_json_object, cw_lists, translate, lists_enabled, + system_language) post_is_sensitive = False if post_json_object['object'].get('sensitive'): @@ -2029,8 +2039,6 @@ def individual_post_as_html(signing_priv_key_pem: str, if not is_pgp_encrypted(content_str): if not is_patch: - # append any edits - content_str += edits_str # Add bold text if bold_reading and \ not displaying_ciphertext and \ @@ -2046,6 +2054,8 @@ def individual_post_as_html(signing_priv_key_pem: str, switch_words(base_dir, nickname, domain, object_content) object_content = html_replace_email_quote(object_content) object_content = html_replace_quote_marks(object_content) + # append any edits + object_content += edits_str else: object_content = content_str else: @@ -2330,7 +2340,8 @@ def html_individual_post(css_cache: {}, instance_title = \ get_config_param(base_dir, 'instanceTitle') - metadata_str = _html_post_metadata_open_graph(domain, original_post_json) + metadata_str = _html_post_metadata_open_graph(domain, original_post_json, + system_language) header_str = html_header_with_external_style(css_filename, instance_title, metadata_str) return header_str + post_str + html_footer() diff --git a/website/EN/index.html b/website/EN/index.html index 6f34ccf6d..b93159608 100644 --- a/website/EN/index.html +++ b/website/EN/index.html @@ -1194,7 +1194,7 @@

An Internet of People, Not Corporate Agendas

- Epicyon is written in Python with a HTML+CSS web interface and uses no javascript which makes display in a web browser very lightweight. It can run as a Progressive Web App on mobile. Just say "no" to boring social media sites packed with generic adverts and zombified corporate influencers. + Epicyon is written in Python with a HTML+CSS web interface and uses no javascript which makes display in a web browser very lightweight. It can run as a Progressive Web App on mobile. Just say "no" to boring social media sites packed with generic adverts and zombified corporate influencers.