alemi/aioappsrv
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

refactor (#5)

Browse source 
* separate dir

* lint

* config

* log unhandled exceptions

* refactor

* refactor webhook check

* cogs cannnot be loaded from another dir

* fix

* fix

* use logger.exception

* Update README.md

* Update README.md

* fix

* Update README.md

* Create README.md

* rm basedir

* cogs

* Update README.md

* bold
This commit is contained in:
git-bruh 2021-03-25 10:48:04 +05:30 committed by GitHub
parent 96ef420f28
commit 8062ba36c6
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 173 additions and 181 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

55
README.md
View file

@ -1,51 +1,22 @@
# matrix-discord-bridge # matrix-discord-bridge
A simple non-puppeting bridge between Matrix and Discord written in Python. A simple bridge between Matrix and Discord written in Python.
## Installation This repository contains two bridges:
* A [puppeting appservice](appservice): The puppeting bridge written with minimal dependencies. Running this requires a self-hosted homeserver.
`pip install -r requirements.txt` * A [non-puppeting bridge](bridge): The non-puppeting bridge written with `matrix-nio` and `discord.py`, most people would want to use this one.
## Usage Check their READMEs for specific information.
* Run `main.py` to generate `config.json`
* Edit `config.json`
```
{
"homeserver": "https://matrix.org",
"username": "@name:matrix.org",
"password": "my-secret-password",
"token": "my-secret-token",
"discord_prefix": "my-command-prefix", # Prefix for Discord commands
"bridge": {
"channel_id": "room_id", # Bridge multiple channels and rooms
"channel_id2": "room_id2"
}
}
```
* Logs are saved to the `bot.log` file in `$PWD`.
* Normal Discord bot functionality like commands can be added to the bot via [cogs](https://discordpy.readthedocs.io/en/latest/ext/commands/cogs.html), example [here](https://gist.github.com/EvieePy/d78c061a4798ae81be9825468fe146be).
* Replace `guild.emojis` with `self.discord_client.emojis` (`Callbacks()`, `process_message()`) to make the Discord bot use emojis from ALL it's guilds.
NOTE: [Privileged Intents](https://discordpy.readthedocs.io/en/latest/intents.html#privileged-intents) must be enabled for your Discord bot.
## Screenshots
TODO
## What Works ## What Works
- [x] Sending messages - [x] Puppeting (Appservice only, regular bridge only uses webhooks on Discord.)
- [x] Discord webhooks (with avatars) - [x] Attachments (Converted to URLs.)
- [x] Attachments (Converted to URLs) - [x] Typing Indicators (Per-user indicators on Appservice, otherwise sent as bot user.)
- [x] Typing status (Not very accurate) - [x] Message redaction
- [x] Redacting messages
- [x] Editing messages
- [x] Replies - [x] Replies
- [x] Bridging multiple channels/rooms - [x] Bridging multiple channels
- [x] `:emote:` in Matrix message converted to Discord emotes - [x] Discord emojis displayed as inline images
- [x] Discord emotes bridged as inline images (Works on Element Web, Fluffychat) - [x] Sending Discord emotes from Matrix (`:emote_name:`)
- [x] Mentioning Discord users via partial username (`@partialname`)

31
bridge/README.md Normal file
View file

@ -0,0 +1,31 @@
## Installation
`pip install -r requirements.txt`
## Usage
* Run `main.py` to generate `config.json`
* Edit `config.json`:
```
{
"homeserver": "https://matrix.org",
"username": "@name:matrix.org",
"password": "my-secret-password", # Matrix password.
"token": "my-secret-token", # Discord bot token.
"discord_cmd_prefix": "my-command-prefix",
"bridge": {
"channel_id": "room_id",
"channel_id2": "room_id2", # Bridge multiple rooms.
},
}
```
This bridge does not use databases for keeping track of bridged rooms to avoid a dependency on persistent storage. This makes it easy to host on something like Heroku with the free tier.
* Logs are saved to the `bridge.log` file in `$PWD`.
* Normal Discord bot functionality like commands can be added to the bot via [cogs](https://discordpy.readthedocs.io/en/latest/ext/commands/cogs.html), example [here](https://gist.github.com/EvieePy/d78c061a4798ae81be9825468fe146be).
NOTE: [Privileged Intents](https://discordpy.readthedocs.io/en/latest/intents.html#privileged-intents) must be enabled for your Discord bot.

242
main.py → bridge/bridge.py
View file

@ -3,9 +3,9 @@ import json
import logging import logging
import os import os
import re import re
import traceback
import sys import sys
import uuid import uuid
import aiofiles import aiofiles
import aiofiles.os import aiofiles.os
import aiohttp import aiohttp
@ -20,8 +20,8 @@ def config_gen(config_file):
"username": "@name:matrix.org", "username": "@name:matrix.org",
"password": "my-secret-password", "password": "my-secret-password",
"token": "my-secret-token", "token": "my-secret-token",
"discord_prefix": "my-command-prefix", "discord_cmd_prefix": "my-command-prefix",
"bridge": {"channel_id": "room_id"} "bridge": {"channel_id": "room_id"},
} }
if not os.path.exists(config_file): if not os.path.exists(config_file):
@ -37,7 +37,6 @@ def config_gen(config_file):
config = config_gen("config.json") config = config_gen("config.json")
message_store = {} message_store = {}
@ -56,17 +55,17 @@ class MatrixClient(nio.AsyncClient):
self.add_callbacks() self.add_callbacks()
def start_discord(self): def start_discord(self):
# Disable everyone and role mentions.
allowed_mentions = discord.AllowedMentions(everyone=False, roles=False)
# Set command prefix for Discord bot.
command_prefix = config["discord_prefix"]
# Intents to fetch members from guild. # Intents to fetch members from guild.
intents = discord.Intents.default() intents = discord.Intents.default()
intents.members = True intents.members = True
self.discord_client = DiscordClient( self.discord_client = DiscordClient(
self, allowed_mentions=allowed_mentions, self,
command_prefix=command_prefix, intents=intents allowed_mentions=discord.AllowedMentions(
everyone=False, roles=False
),
command_prefix=config["discord_cmd_prefix"],
intents=intents,
) )
self.bg_task = self.loop.create_task( self.bg_task = self.loop.create_task(
@ -78,8 +77,7 @@ class MatrixClient(nio.AsyncClient):
self.add_event_callback( self.add_event_callback(
callbacks.message_callback, callbacks.message_callback,
(nio.RoomMessageText, nio.RoomMessageMedia, (nio.RoomMessageText, nio.RoomMessageMedia, nio.RoomMessageEmote),
nio.RoomMessageEmote)
) )
self.add_event_callback( self.add_event_callback(
@ -107,16 +105,12 @@ class MatrixClient(nio.AsyncClient):
await f.write(emote) await f.write(emote)
async with aiofiles.open(emote_file, "rb") as f: async with aiofiles.open(emote_file, "rb") as f:
resp, maybe_keys = await self.upload( resp, maybe_keys = await self.upload(f, content_type=content_type)
f, content_type=content_type
)
await aiofiles.os.remove(emote_file) await aiofiles.os.remove(emote_file)
if type(resp) != nio.UploadResponse: if type(resp) != nio.UploadResponse:
self.logger.warning( self.logger.warning(f"Failed to upload emote {emote_id}")
f"Failed to upload emote {emote_id}"
)
return return
self.uploaded_emotes[emote_id] = resp.content_uri self.uploaded_emotes[emote_id] = resp.content_uri
@ -125,18 +119,18 @@ class MatrixClient(nio.AsyncClient):
async def get_fmt_body(self, body, emotes): async def get_fmt_body(self, body, emotes):
replace_ = [ replace_ = [
# Code blocks # Bold.
("```", "<pre><code>", "</code></pre>"), ("**", "<strong>", "</strong>"),
# Spoilers # Code blocks.
("||", "<span data-mx-spoiler>", "</span>"), ("```", "<pre><code>", "</code></pre>"),
# Strikethrough # Spoilers.
("~~", "<del>", "</del>") ("||", "<span data-mx-spoiler>", "</span>"),
] # Strikethrough.
("~~", "<del>", "</del>"),
]
for replace in replace_: for replace in replace_:
for i in range(body.count(replace[0])): for i in range(1, body.count(replace[0]) + 1):
i += 1
if i % 2: if i % 2:
body = body.replace(replace[0], replace[1], 1) body = body.replace(replace[0], replace[1], 1)
else: else:
@ -147,77 +141,85 @@ class MatrixClient(nio.AsyncClient):
if emote_: if emote_:
emote = f":{emote}:" emote = f":{emote}:"
body = body.replace( body = body.replace(
emote, f"""<img alt=\"{emote}\" title=\"{emote}\" \ emote,
height=\"32\" src=\"{emote_}\" data-mx-emoticon />""" f"""<img alt=\"{emote}\" title=\"{emote}\" \
height=\"32\" src=\"{emote_}\" data-mx-emoticon />""",
) )
return body return body
async def message_send(self, message, channel_id, emotes, async def message_send(
reply_id=None, edit_id=None): self, message, channel_id, emotes, reply_id=None, edit_id=None
):
room_id = config["bridge"][str(channel_id)] room_id = config["bridge"][str(channel_id)]
content = { content = {
"body": message, "body": message,
"format": "org.matrix.custom.html", "format": "org.matrix.custom.html",
"formatted_body": await self.get_fmt_body(message, emotes), "formatted_body": await self.get_fmt_body(message, emotes),
"msgtype": "m.text" "msgtype": "m.text",
} }
if reply_id: if reply_id:
reply_event = await self.room_get_event( reply_event = await self.room_get_event(room_id, reply_id)
room_id, reply_id
)
reply_event = reply_event.event reply_event = reply_event.event
content["m.relates_to"] = {"m.in_reply_to": {"event_id": reply_id}} content = {
**content,
content["formatted_body"] = f"""<mx-reply><blockquote>\ "m.relates_to": {"m.in_reply_to": {"event_id": reply_id}},
"formatted_body": f"""<mx-reply><blockquote>\
<a href="https://matrix.to/#/{room_id}/{reply_id}">In reply to</a>\ <a href="https://matrix.to/#/{room_id}/{reply_id}">In reply to</a>\
<a href="https://matrix.to/#/{reply_event.sender}">{reply_event.sender}</a>\ <a href="https://matrix.to/#/{reply_event.sender}">{reply_event.sender}</a>\
<br>{reply_event.body}</blockquote></mx-reply>{content["formatted_body"]}""" <br>{reply_event.body}</blockquote></mx-reply>{content["formatted_body"]}""",
if edit_id:
content["body"] = f" * {content['body']}"
content["m.relates_to"] = {
"event_id": edit_id, "rel_type": "m.replace"
} }
content["m.new_content"] = { if edit_id:
"body": content["body"], content = {
"formatted_body": content["formatted_body"], **content,
"format": content["format"], "body": f" * {content['body']}",
"msgtype": content["msgtype"] "formatted_body": f" * {content['formatted_body']}",
"m.relates_to": {"event_id": edit_id, "rel_type": "m.replace"},
"m.new_content": {**content},
} }
message = await self.room_send( message = await self.room_send(
room_id=room_id, room_id=room_id, message_type="m.room.message", content=content
message_type="m.room.message",
content=content
) )
return message.event_id return message.event_id
async def message_redact(self, message, channel_id): async def message_redact(self, message, channel_id):
await self.room_redact( await self.room_redact(
room_id=config["bridge"][str(channel_id)], room_id=config["bridge"][str(channel_id)], event_id=message
event_id=message
) )
async def webhook_send(self, author, avatar, message, async def webhook_send(
event_id, channel_id, embed=None): self, author, avatar, message, event_id, channel_id, embed=None
):
channel = self.discord_client.channel_store[channel_id] channel = self.discord_client.channel_store[channel_id]
# Recreate hook if it was deleted. hook_name = "matrix_bridge"
hook = await self.discord_client.hook_create(channel)
hook = self.discord_client.webhook_cache.get(str(channel.id))
if not hook:
hooks = await channel.webhooks()
hook = discord.utils.get(hooks, name=hook_name)
if not hook:
hook = await channel.create_webhook(name=hook_name)
self.discord_client.webhook_cache[str(channel.id)] = hook
# Username must be between 1 and 80 characters in length, # Username must be between 1 and 80 characters in length,
# 'wait=True' allows us to store the sent message. # 'wait=True' allows us to store the sent message.
try: try:
hook = await hook.send( hook = await hook.send(
username=author[:80], avatar_url=avatar, username=author[:80],
content=message, embed=embed, wait=True avatar_url=avatar,
content=message,
embed=embed,
wait=True,
) )
message_store[event_id] = hook message_store[event_id] = hook
@ -232,7 +234,7 @@ class DiscordClient(discord.ext.commands.Bot):
self.channel_store = {} self.channel_store = {}
self.webhook_ids = set() self.webhook_cache = {}
self.ready = asyncio.Event() self.ready = asyncio.Event()
@ -251,37 +253,20 @@ class DiscordClient(discord.ext.commands.Bot):
cog = f"cogs.{cog[:-3]}" cog = f"cogs.{cog[:-3]}"
self.load_extension(cog) self.load_extension(cog)
async def hook_create(self, channel):
hook_name = "matrix_bridge"
hooks = await channel.webhooks()
# Check if webhook exists.
hook = discord.utils.get(hooks, name=hook_name)
if not hook:
hook = await channel.create_webhook(name=hook_name)
self.webhook_ids.add(hook.id)
return hook
async def to_return(self, channel_id, message=None): async def to_return(self, channel_id, message=None):
await self.matrix_client.ready.wait() await self.matrix_client.ready.wait()
if str(channel_id) not in config["bridge"].keys(): return str(channel_id) not in config["bridge"].keys() or (
return True message
and message.webhook_id
if message: in [hook.id for hook in self.webhook_cache.values()]
if message.webhook_id in self.webhook_ids: )
return True
async def on_ready(self): async def on_ready(self):
for channel in config["bridge"].keys(): for channel in config["bridge"].keys():
channel_ = self.get_channel(int(channel)) channel_ = self.get_channel(int(channel))
self.channel_store[channel] = channel_ self.channel_store[channel] = channel_
await self.hook_create(channel_)
self.ready.set() self.ready.set()
async def on_message(self, message): async def on_message(self, message):
@ -294,8 +279,10 @@ class DiscordClient(discord.ext.commands.Bot):
content = await self.process_message(message) content = await self.process_message(message)
matrix_message = await self.matrix_client.message_send( matrix_message = await self.matrix_client.message_send(
content[0], message.channel.id, content[0],
reply_id=content[1], emotes=content[2] message.channel.id,
reply_id=content[1],
emotes=content[2],
) )
message_store[message.id] = matrix_message message_store[message.id] = matrix_message
@ -309,8 +296,10 @@ class DiscordClient(discord.ext.commands.Bot):
# Edit message only if it can be looked up in the cache. # Edit message only if it can be looked up in the cache.
if before.id in message_store: if before.id in message_store:
await self.matrix_client.message_send( await self.matrix_client.message_send(
content[0], after.channel.id, content[0],
edit_id=message_store[before.id], emotes=content[2] after.channel.id,
edit_id=message_store[before.id],
emotes=content[2],
) )
async def on_message_delete(self, message): async def on_message_delete(self, message):
@ -371,8 +360,12 @@ class Callbacks(object):
def get_channel(self, room): def get_channel(self, room):
channel_id = next( channel_id = next(
(channel_id for channel_id, room_id in config["bridge"].items() (
if room_id == room.room_id), None channel_id
for channel_id, room_id in config["bridge"].items()
if room_id == room.room_id
),
None,
) )
return channel_id return channel_id
@ -380,10 +373,11 @@ class Callbacks(object):
async def to_return(self, room, event): async def to_return(self, room, event):
await self.matrix_client.discord_client.ready.wait() await self.matrix_client.discord_client.ready.wait()
if room.room_id not in config["bridge"].values() or \ return (
event.sender == self.matrix_client.user or \ room.room_id not in config["bridge"].values()
not self.matrix_client.listen: or event.sender == self.matrix_client.user
return True or not self.matrix_client.listen
)
async def message_callback(self, room, event): async def message_callback(self, room, event):
message = event.body message = event.body
@ -418,7 +412,8 @@ class Callbacks(object):
await webhook_message.edit(content=edited_content) await webhook_message.edit(content=edited_content)
# Handle exception if edited message was deleted on Discord. # Handle exception if edited message was deleted on Discord.
except ( except (
discord.errors.NotFound, discord.errors.HTTPException discord.errors.NotFound,
discord.errors.HTTPException,
) as e: ) as e:
self.matrix_client.logger.warning( self.matrix_client.logger.warning(
f"Failed to edit message {edited_event}: {e}" f"Failed to edit message {edited_event}: {e}"
@ -429,9 +424,11 @@ class Callbacks(object):
pass pass
try: try:
if content_dict["m.relates_to"]["m.in_reply_to"]["event_id"] in \ if (
message_store.values(): content_dict["m.relates_to"]["m.in_reply_to"]["event_id"]
# Remove the first occurance of our bot's username if replying. in message_store.values()
):
# Remove the first occurence of our bot's username if replying.
# > <@discordbridge:something.org> [discord user] # > <@discordbridge:something.org> [discord user]
message = message.replace(f"<{config['username']}>", "", 1) message = message.replace(f"<{config['username']}>", "", 1)
except KeyError: except KeyError:
@ -489,13 +486,14 @@ class Callbacks(object):
pass pass
async def typing_callback(self, room, event): async def typing_callback(self, room, event):
if not room.typing_users \ if (
or room.room_id not in config["bridge"].values(): not room.typing_users
return or (
len(room.typing_users) == 1
# Return if the event is sent by our bot. and self.matrix_client.user in room.typing_users
if len(room.typing_users) == 1 and \ )
self.matrix_client.user in room.typing_users: or room.room_id not in config["bridge"].values()
):
return return
# Get the corresponding Discord channel. # Get the corresponding Discord channel.
@ -506,8 +504,8 @@ class Callbacks(object):
return return
async def process_message(self, message, channel_id): async def process_message(self, message, channel_id):
mentions = re.findall(r"(^|\s)(@(\w*))", message)
emotes = re.findall(r":(\w*):", message) emotes = re.findall(r":(\w*):", message)
mentions = re.findall(r"(@(\w*))", message)
# Get the guild from channel ID. # Get the guild from channel ID.
guild = self.discord_client.channel_store[channel_id].guild guild = self.discord_client.channel_store[channel_id].guild
@ -522,15 +520,15 @@ class Callbacks(object):
if emote_: if emote_:
message = message.replace(f":{emote}:", str(emote_)) message = message.replace(f":{emote}:", str(emote_))
# mentions = [('', '@name', 'name'), (' ', '@', '')] # mentions = [('@name', 'name'), ('@', '')]
for mention in mentions: for mention in mentions:
# Don't fetch member if mention is empty. # Don't fetch member if mention is empty.
# Single "@" without any name. # Single "@" without any name.
if mention[2]: if mention[1]:
member = await guild.query_members(query=mention[2]) member = await guild.query_members(query=mention[1])
if member: if member:
# Get first result. # Get first result.
message = message.replace(mention[1], member[0].mention) message = message.replace(mention[0], member[0].mention)
return message return message
@ -538,17 +536,17 @@ class Callbacks(object):
async def main(): async def main():
logging.basicConfig( logging.basicConfig(
level=logging.INFO, level=logging.INFO,
format="%(asctime)s %(name)s:%(levelname)s: %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
handlers=[ handlers=[
logging.FileHandler("bot.log"), logging.FileHandler("bridge.log"),
logging.StreamHandler() logging.StreamHandler(),
] ],
) )
retry = 2 retry = 2
matrix_client = MatrixClient( matrix_client = MatrixClient(config["homeserver"], config["username"])
config["homeserver"], config["username"]
)
while True: while True:
resp = await matrix_client.login(config["password"]) resp = await matrix_client.login(config["password"])
@ -563,9 +561,7 @@ async def main():
try: try:
await matrix_client.sync(full_state=True) await matrix_client.sync(full_state=True)
except Exception: except Exception:
matrix_client.logger.error( matrix_client.logger.exception("Initial sync failed!")
f"Initial sync failed!\n{traceback.format_exc()}"
)
return False return False
try: try:
@ -576,9 +572,8 @@ async def main():
await matrix_client.sync_forever(timeout=30000, full_state=True) await matrix_client.sync_forever(timeout=30000, full_state=True)
except Exception: except Exception:
matrix_client.logger.error( matrix_client.logger.exception(
f"Unknown exception occured\n{traceback.format_exc()}\n" f"Unknown exception occured, retrying in {retry} seconds..."
f"Retrying in {retry} seconds..."
) )
# Clear "ready" status. # Clear "ready" status.
@ -593,5 +588,6 @@ async def main():
await matrix_client.close() await matrix_client.close()
return False return False
if __name__ == "__main__": if __name__ == "__main__":
asyncio.run(main()) asyncio.run(main())

0
cogs/.keep → bridge/cogs/.keep
View file

2
requirements.txt → bridge/requirements.txt
View file

@ -1,3 +1,3 @@
aiofiles==0.6.0 aiofiles==0.6.0
discord.py==1.6.0 discord.py==1.6.0
matrix-nio==0.16.0 matrix-nio==0.17.0

24
utils/migrate_emotes.py → misc/migrate_emotes.py
View file

@ -17,7 +17,7 @@ def config_gen(config_file):
"username": "@name:matrix.org", "username": "@name:matrix.org",
"password": "my-secret-password", "password": "my-secret-password",
"token": "my-secret-token", "token": "my-secret-token",
"migrate": {"guild_id": "room_id"} "migrate": {"guild_id": "room_id"},
} }
if not os.path.exists(config_file): if not os.path.exists(config_file):
@ -66,16 +66,12 @@ class MatrixClient(nio.AsyncClient):
await f.write(emote_) await f.write(emote_)
async with aiofiles.open(emote_file, "rb") as f: async with aiofiles.open(emote_file, "rb") as f:
resp, maybe_keys = await self.upload( resp, maybe_keys = await self.upload(f, content_type=content_type)
f, content_type=content_type
)
await aiofiles.os.remove(emote_file) await aiofiles.os.remove(emote_file)
if type(resp) != nio.UploadResponse: if type(resp) != nio.UploadResponse:
self.logger.warning( self.logger.warning(f"Failed to upload {emote_name}")
f"Failed to upload {emote_name}"
)
return return
self.logger.info(f"Uploaded {emote_name}") self.logger.info(f"Uploaded {emote_name}")
@ -101,9 +97,7 @@ class MatrixClient(nio.AsyncClient):
resp = await self.room_put_state(room_id, event_type, content) resp = await self.room_put_state(room_id, event_type, content)
if type(resp) == nio.RoomPutStateError: if type(resp) == nio.RoomPutStateError:
self.logger.warning( self.logger.warning(f"Failed to send emote state: {resp}")
f"Failed to send emote state: {resp}"
)
class DiscordClient(discord.Client): class DiscordClient(discord.Client):
@ -115,8 +109,8 @@ class DiscordClient(discord.Client):
) )
self.bg_task = self.loop.create_task( self.bg_task = self.loop.create_task(
self.log_exceptions(self.matrix_client) self.log_exceptions(self.matrix_client)
) )
self.logger = logging.getLogger("discord_logger") self.logger = logging.getLogger("discord_logger")
@ -138,9 +132,9 @@ class DiscordClient(discord.Client):
f"Guild: {emote_guild.name} Room: {emote_room}" f"Guild: {emote_guild.name} Room: {emote_room}"
) )
await asyncio.gather(*map( await asyncio.gather(
self.matrix_client.upload_emote, emote_guild.emojis *map(self.matrix_client.upload_emote, emote_guild.emojis)
)) )
self.logger.info("Sending state event to room...") self.logger.info("Sending state event to room...")