Added plugin install API

It is now specified how a plugin should be installed. Furthermore, a
"super user" is added to the install script and configuration files.

README.md

@@ -17,7 +17,7 @@ This bot was orginally created to serve two purposes. Firstly, and mainly, it se
 ## Installation & Requirements:
 First, clone this repository:
 ```
-git clone https://git.dennispotter.eu/Dennis/matrix-chatbot
+git clone https://git.dennispotter.eu/Dennis/plugable-matrix-bot
 ```
 In order for the bot framework to work, Python 3, [virtualenv](https://virtualenv.pypa.io/en/latest/), and [pip](https://pypi.org/project/pip/) must be installed on your computer or server.
 
@@ -62,11 +62,13 @@ The bot has different plugins that can be activated by sending `[Keyword in bot'
 To interact with rooms, the [Matrix Python SDK](http://matrix-org.github.io/matrix-python-sdk/<Paste>) can be used.
 
 #### Mandatory/recommended files
-To create a plugin, a few mandatory or recommended files must be present. The tree below shows the general structure of a plugin. A new plugin must be placed in a directory with the same name. The main class (more on that later) of the plugin must be defined in a Python file with the same name within that directory. The event may not use bot's main configuration file. All configuration must be placed in a separate `config.json`.
+To create a plugin, a few files must or can be present. The tree below shows the general structure of a plugin. A new plugin must be placed in a directory that has the name of that plugin. The main class (more on that later) of the plugin must be defined in a Python file with that same name. The plugin may not use bot's main configuration file. All configuration must be placed in a separate `config.json`.
 
 A subdirectory `messages` is used to store messages. Within this directory, a file `help` may be present. The content of this file must be returned with a method of the plugin class (more on that later). Furthermore, for every language a file `messages.<language>.json` with all messages should exist.
 
-The help file is not mandatory. If no help message for a plugin should appear when sending a help request to the bot, this file can be ommitted. The Plugin's `help()` method should return 0 in that case.
+The optional file `requirements.txt` is used to define all dependencies. The framework will automatically install them before the first time the plugin is used. This happens even *before* the `setup()` function is invoked (more on that later).
+
+Although recommended, the help file is also not mandatory. If no help message for a plugin should appear when sending a help request to the bot, this file can be ommitted. The Plugin's `help()` method should return 0 in that case.
 ```
 .
 └── plugins
@@ -75,6 +77,7 @@ The help file is not mandatory. If no help message for a plugin should appear wh
     │   ├── <plugin1>.py
     │   ├── README.md
     │   ├── config.json
+    │   ├── requirements.txt
     │   └── messages
     │       ├── messages.<language>.json
     │       └── help
@@ -103,7 +106,7 @@ The code belows shows an example of such a help function.
 The code below shows the template for a simple plugin with a few features. The name of the class of every plugin must be `Plugin`.
 
 * It has a method `__init__()` which is executed every time the bot is started. Usually, this should load the configuration file, it should set the sensitivity on which it should execute certain callback functions, it should save the parent object, start additional threads, and execute the setup.
-* It has a method `setup()` which is only executed once (ever). This can be used, for example, to create [SQLite](https://sqlite.org) tables. All data should be saved in subdirectory for that plugin in `data`. See the example below, where `<plugin1>` initiates an SQLite database.
+* The python file may contain a function `setup()`. (**Thus not a method of the class!**) The framework makes sure that this function is executed once, the first time the plugin is used. `setup()` can be used, for example, to create [SQLite](https://sqlite.org) tables. All data should be saved in subdirectory for that plugin in `data`. See the example below, where `<plugin1>` initiates an SQLite database.
 
 ```
 .
@@ -154,18 +157,6 @@ class Plugin:
         self.thread1 = threading.Thread(target=self.thread1_method)
         self.thread1.start()
 
-        self.setup()
-
-    def setup(self):
-        """This function only runs once, ever. It installs the plugin"""
-
-        try:
-            # Install some stuff, if not already installed
-        except sqlite3.OperationalError:
-            # For example, when trying to initialize an SQLite DB
-            # which already exists, sqlite3.OperationalError is thrown
-            pass
-
     def thread1_method(self):
         """This function continuously loops"""
 
@@ -173,7 +164,7 @@ class Plugin:
             # Do something. For example, call other methods.
 
             # Sleep
-            time.sleep(self.config['update_time_span'] * 60)
+            time.sleep(60)
 
     def callback1(self, room, event):
         """Act on first sensitivity"""
@@ -187,6 +178,11 @@ class Plugin:
 
     def help(self):
         return open(HELP_LOCATION, mode="r").read()
+
+def setup():
+    """This function only runs once, ever. It installs the plugin"""
+
+    # Install some stuff, e.g., initialize an SQLite database
 ```
 
 #### Additional tips

config.json.template

@@ -1,8 +1,8 @@
 {
     "bot_credentials": {
-        "username": "${uservar}",
+        "username": "${username}",
-        "password": "${passvar}",
+        "password": "${password}",
-        "server": "${servervar}"
+        "server": "${server}"
     },
 
     "character": {
@@ -13,5 +13,13 @@
 
     "plugins": [
         "hello"
+    ],
+
+    "triggers": {
+        "help": "Bot help"
+    },
+
+    "super_users": [
+        "${superuser}"
     ]
 }

install.sh

@@ -1,12 +1,12 @@
 #!/bin/bash
 #
-# Installation script for matrix-chatbot
+# Installation script for plugable-matrix-bot
 #
 # @author Dennis Potter <dennis@dennispotter.eu>
 # @copyright 2019, Dennis Potter
 # @license GNU General Public License (version 3)
 #
-# matrix-chatbot
+# plugable-matrix-bot
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -89,6 +89,10 @@ echo "forms the perfect base for further development."
 echo ""
 echo "In the following steps, an _existing_ user must be provided. If you did not"
 echo "yet create a user for your bot, please do so now!"
+echo ""
+echo "The installation procedure will also ask you to define a super user. A super"
+echo "user is a normal user that can manage the bot. This SU is passed on to all"
+echo "plugins."
 echo "############################################################################"
 echo ""
 
@@ -101,13 +105,15 @@ if [[ -f $DIR/config.json ]]; then
     fi
 fi
 
-read -p 'Matrix homeserver: ' servervar
+read -p 'Matrix homeserver: ' server
-read -p 'Matrix username: ' uservar
+read -p 'Matrix username: ' username
-read -p 'Matrix password: ' passvar
+read -p 'Matrix password: ' password
+read -p 'Super user: ' superuser
 
-sed -e "s/\${servervar}/${servervar}/"\
+sed -e "s/\${server}/${server}/"\
-    -e "s/\${uservar}/${uservar}/"\
+    -e "s/\${username}/${username}/"\
-    -e "s/\${passvar}/${passvar}/"\
+    -e "s/\${password}/${password}/"\
+    -e "s/\${superuser}/${superuser}/"\
     $DIR/config.json.template > $DIR/config.json
 
 #################################################################
@@ -120,9 +126,9 @@ fi
 sudo useradd -r mchatbot
 
 sed -e "s/\${DIR}/${DIR}/"\
-    $DIR/matrix-chatbot.service.template > $DIR/matrix-chatbot.service
+    $DIR/plugable-matrix-bot.service.template > $DIR/plugable-matrix-bot.service
 
-sudo mv $DIR/matrix-chatbot.service /etc/systemd/system/matrix-chatbot.service
+sudo mv $DIR/plugable-matrix-bot.service /etc/systemd/system/plugable-matrix-bot.service
 
-sudo systemctl enable matrix-chatbot.service
+sudo systemctl enable plugable-matrix-bot.service
-sudo systemctl start matrix-chatbot.service
+sudo systemctl start plugable-matrix-bot.service

matrix_bot_api/matrix_bot_api.py

@@ -3,6 +3,10 @@ import re
 import os
 import sys
 import importlib
+import sqlite3
+import datetime as dt
+
+from pip._internal import main as pipmain
 
 from matrix_client.client import MatrixClient
 from matrix_client.api import MatrixRequestError, MatrixHttpApi
@@ -11,6 +15,7 @@ from matrix_client.user import User
 from matrix_bot_api.mregex_handler import MRegexHandler
 
 HELP_LOCATION = os.path.join(os.path.dirname(__file__), 'help')
+DATA_LOCATION = os.path.join(os.path.dirname(__file__), '../data/bot.db')
 
 private_message = "Hey {}! Ik heb je even een privébericht gestuurd 🙂"
 
@@ -19,16 +24,17 @@ def eprint(*args, **kwargs):
     print(*args, file=sys.stderr, **kwargs)
 
 class MatrixBotAPI:
-    # rooms    - List of rooms ids to operate in, or None to accept all rooms
+    # rooms - List of rooms ids to operate in, or None to accept all rooms
     def __init__(self, config, rooms=None):
-        self.username = config['bot_credentials']['username']
+        self.config = config
+        self.username = self.config['bot_credentials']['username']
 
         # Authenticate with given credentials
-        self.client = MatrixClient(config['bot_credentials']['server'])
+        self.client = MatrixClient(self.config['bot_credentials']['server'])
         try:
             self.token = self.client.login_with_password(
-                config['bot_credentials']['username'],
+                self.config['bot_credentials']['username'],
-                config['bot_credentials']['password'])
+                self.config['bot_credentials']['password'])
         except MatrixRequestError as e:
             print(e)
             if e.code == 403:
@@ -38,7 +44,7 @@ class MatrixBotAPI:
             traceback.print_exc()
 
         self.api = MatrixHttpApi(
-                config['bot_credentials']['server'],
+                self.config['bot_credentials']['server'],
                 token=self.token)
 
         # Store allowed rooms
@@ -67,56 +73,135 @@ class MatrixBotAPI:
         # of all plugins
         self.cancel = False
 
-        # Store empty list of plugins
+        # Run setup
-        self.plugins = []
+        self.setup()
 
+        # Add plugins
+        self.add_plugins()
+
+        # Add callback for help function
+        self.help_handler = MRegexHandler(
+            self.config['triggers']['help'], self.help)
+        self.add_handler(self.help_handler)
+
+    def add_plugins(self):
+        # Store empty list of plugins
+        self.plugin_objects = []
 
         # Try to import plugins. All plugins must be located in the
         # ./plugins directory
         modules = []
 
-        try:
+        #try:
-            for plugin in config['plugins']:
+        for plugin in self.config['plugins']:
-                modules.append(
+            modules.append(
-                    importlib.import_module(
+                importlib.import_module(
-                        "plugins.{0}.{0}".format(plugin), package = None))
+                    "plugins.{0}.{0}".format(plugin), package = None))
-        except:
+        #except:
-            eprint("Importing one or more of the plugins did not go well!")
+        #    eprint("Importing one or more of the plugins did not go well!")
-            sys.exit()
+        #    sys.exit()
 
         # Loop through the available modules and add their handler variables
         for i, module in enumerate(modules):
-            # Create new instance of plugin and append to plugins array
+            # Check if module is already installed
-            self.plugins.append(module.Plugin(self))
+            self.setup_plugin(module)
+
+            # Create new instance of plugin and append to plugin_objects array
+            self.plugin_objects.append(module.Plugin(self))
 
             # Add handler of newly created instance to bot
-            self.add_handler(self.plugins[i].handler)
+            self.add_handler(self.plugin_objects[i].handler)
 
-        # Run setup
+    def setup_plugin(self, module):
-        self.setup(config)
+        """Check in an SQLite database if the plugin is already installed.
+        If this is not the case, install all required packages and run the
+        plugin's setup method.
+        """
+        module_name = module.__name__.split('.')[-1]
 
-        # Add callback for help function
+        # Open SQLite database
-        self.help_handler = MRegexHandler("Peter help", self.help)
+        sqlite_db = sqlite3.connect(DATA_LOCATION)
-        self.add_handler(self.help_handler)
 
-    def setup(self, config):
+        sqlite_cursor = sqlite_db.cursor()
+
+        # Define query to SELECT event from SQLite DB
+        select_sql = f"""SELECT module_name
+                         FROM plugins
+                         WHERE module_name = "{module_name}" """
+
+        sqlite_db = sqlite3.connect(DATA_LOCATION)
+        sqlite_cursor = sqlite_db.cursor()
+        sqlite_cursor.execute(select_sql)
+
+        if sqlite_cursor.fetchall():
+            # Do nothing, this was already installed
+            print(f"Check: {module_name} already installed.")
+        else:
+            # This appears to be new, insert it, process requirements.txt
+            # and run the plugin's setup file.
+            print(f"Running installation of {module_name}.")
+
+            # Install requirements
+            pipmain(
+                ['install',
+                 '-r',
+                  f'plugins/{module_name}/requirements.txt'])
+
+            # Run module's install method
+            try:
+                module.setup()
+            except:
+                print(f"{module_name} did not specify setup(). Skipping...")
+
+            # Save in database that we installed this plugin
+            datetime_added = dt.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+
+            insert_sql = f"""INSERT INTO plugins(module_name, datetime_added)
+                             VALUES('{module_name}', '{datetime_added}')"""
+
+            sqlite_cursor.execute(insert_sql)
+            sqlite_db.commit()
+            sqlite_db.close()
+
+            # Tell the user how awesome we are
+            print(f"Successfully installed {module_name}.")
+
+
+
+    def setup(self):
         """This function only runs once, ever. It initializes an SQLite
-        database, sets the bot's avatar, and name"""
+        database, sets the bot's avatar, and name
+        """
 
         try:
             # Try to make new directory
             os.mkdir("data")
 
+            # Create SQLite database
+            # Define query to INSERT event table to SQLite DB
+            sql = """CREATE TABLE 'plugins' (
+                     'id' INTEGER PRIMARY KEY AUTOINCREMENT,
+                     'datetime_added' DATTIME,
+                     'module_name' TEXT);"""
+
+            # Open, execute, commit, and close SQLite database
+            sqlite_db = sqlite3.connect(DATA_LOCATION)
+            sqlite_cursor = sqlite_db.cursor()
+            sqlite_cursor.execute(sql)
+            sqlite_db.commit()
+            sqlite_db.close()
+
             # Get user instance
             self.user = self.client.get_user(self.username)
 
-            # Set username
+            ## Set username
-            self.user.set_display_name(config['character']['name'])
+            self.user.set_display_name(self.config['character']['name'])
 
-            # Open, upload, and set avatar
+            ## Open, upload, and set avatar
-            f = open(config['character']['avatar'], mode="rb")
+            f = open(self.config['character']['avatar'], mode="rb")
 
-            avatar = self.client.upload(f, config['charactar']['avatar_mime'])
+            avatar = self.client.upload(
+                f, self.config['charactar']['avatar_mime'])
             self.user.set_avatar_url(avatar)
 
         except FileExistsError:
@@ -159,7 +244,7 @@ class MatrixBotAPI:
 
         help_text = open(HELP_LOCATION, mode="r").read()
 
-        for plugin in self.plugins:
+        for plugin in self.plugin_objects:
             try:
                 help_text += plugin.help()
             except TypeError:

plugins/admidio_events/admidio_events.py

@@ -53,40 +53,10 @@ class Plugin:
         # Save parent bot
         self.bot = bot
 
-        # Run one time setup
-        self.setup()
-
         # Start thread to check events
         self.event_thread = threading.Thread(target=self.check_new_event_thread)
         self.event_thread.start()
 
-    def setup(self):
-        """This function only runs once, ever. It initializes the necessary
-        SQLite tables."""
-
-        try:
-            # Try to make a new directory
-            os.mkdir(EVENTS_DATA_DIR)
-
-            # Define query to INSERT event table to SQLite DB
-            sql = """CREATE TABLE 'events' (
-                     'dat_id' integer,
-                     'datetime_posted' datetime);"""
-
-            # Open, execute, commit, and close SQLite database
-            sqlite_db = sqlite3.connect(DATA_LOCATION)
-            sqlite_cursor = sqlite_db.cursor()
-            sqlite_cursor.execute(sql)
-            sqlite_db.commit()
-            sqlite_db.close()
-
-        except sqlite3.OperationalError:
-            # Table already exists, do nothing
-            pass
-        except FileExistsError:
-            # Directory already exists, do nothing
-            pass
-
     def connect_database(self):
         # Connect to database
         return mysql.connect(
@@ -210,7 +180,7 @@ class Plugin:
 
         if not number_events:
             number_events = 10
-        if number_events <= 0 or number_events > 100:
+        elif number_events <= 0 or number_events > 100:
             room.send_text(self.messages['list_event_num_err'])
             number_events = 10
 
@@ -272,3 +242,23 @@ class Plugin:
 
     def help(self):
         return open(HELP_LOCATION, mode="r").read()
+
+def setup():
+    """This function runs only, ever. It initializes the necessary
+    SQLite tables."""
+
+    # Try to make a new directory
+    os.mkdir(EVENTS_DATA_DIR)
+
+    # Define query to INSERT event table to SQLite DB
+    sql = """CREATE TABLE 'events' (
+             'dat_id' integer,
+             'datetime_posted' datetime);"""
+
+    # Open, execute, commit, and close SQLite database
+    sqlite_db = sqlite3.connect(DATA_LOCATION)
+    sqlite_cursor = sqlite_db.cursor()
+    sqlite_cursor.execute(sql)
+    sqlite_db.commit()
+    sqlite_db.close()
+

plugins/admidio_events/requirements.txt

@@ -0,0 +1 @@
+mysqlclient

plugins/hello/hello.py

@@ -13,13 +13,16 @@ class Plugin:
         # Define sensitivity
         self.handler = []
 
-        self.handler.append(MRegexHandler("Hello bot", self.info_callback))
+        self.handler.append(MRegexHandler("Hello bot", self.hello_callback))
 
         # Save parent bot
         self.bot = bot
 
-    def info_callback(self, room, event):
+    def hello_callback(self, room, event):
         room.send_text(f"Hello {event['sender']}!")
 
     def help(self):
         return open(HELP_LOCATION, mode="r").read()
+
+def setup():
+    print("This function runs once, when the plugin is used the 1st time.")

plugins/hello/requirements.txt

@@ -0,0 +1 @@
+argh