Installation and Getting Started¶
Local Development¶
git clone https://github.com/amyxzhang/policykit.git
cd policykit/policykit
pip install --upgrade pip
pip install -r requirements.txt
cd policykit
cp .env.example .env
.env
file such as the DJANGO_SECRET_KEY
and SERVER_URL
. For local development, all you need to do is set DEBUG=true
.python manage.py runserver
DATABASES
field in settings.py
.python manage.py migrate
Open PolicyKit in the browser at http://localhost:8000/main. At this point, you won’t be able to log in because PolicyKit currently only supports sign-in via external auth providers (Slack, Discord, Reddit, and Discourse). There is an open issue to support logging in without any third-party platform: #514.
To log in to PolicyKit, you’ll need to install it on a dev server and set up at least 1 of the auth-enabled integrations.
Running PolicyKit on a Server¶
Add PolicyKit to the server by uploading the codebase or using
git clone
.Follow this guide to install Python3 and to create a virtual environment for PolicyKit.
Install the requirements to the virtual environment with
pip install -r requirements.txt
.Finish the earlier guide to setting up PolicyKit.
Make the following additional changes to
.env
:Set the
DJANGO_SECRET_KEY
field. Generate a key with this command:python manage.py shell -c 'from django.core.management import utils; print(utils.get_random_secret_key())'
Set the
SERVER_URL
field.Set the
ALLOWED_HOSTS
field to point to your host.Make sure
DEBUG
is empty or set to false.You can leave the platform integration API keys/secrets empty for now. Follow the instructions under “Set up Integrations” to set up each integration.
If you want to use a database other than dbsqlite3, or if you want to change the database path, update the
DATABASES
object insettings.py
.Next, run the following command to collect static files into a
static/
folder:
python manage.py collectstatic
Deploy with Apache web server¶
Now that you have PolicyKit installed on your server, you can deploy it on Apache web server. Make sure you have a domain dedicated to Policykit that is pointing to your server’s IP address.
Note
In the remaining examples, make sure to substitute the following values:
$POLICYKIT_REPO
is the path to your policykit repository root. (/policykit
)
$POLICYKIT_ENV
is the path to your policykit virtual environment. (/environments/policykit_env
)
$SERVER_NAME
is your server name. (policykit.mysite.com
)
Install apache2
sudo apt-get install apache2 libapache2-mod-wsgi-py3
Create a new apache2 config file:
cd /etc/apache2/sites-available # replace SERVER_NAME (ie policykit.mysite.com.conf) cp default-ssl.conf SERVER_NAME.conf
Edit the config file to look like this:
<IfModule mod_ssl.c> <VirtualHost _default_:443> ServerName $SERVER_NAME ServerAdmin webmaster@localhost Alias /static $POLICYKIT_REPO/policykit/static <Directory $POLICYKIT_REPO/policykit/static> Require all granted </Directory> # Grant access to wsgi.py file. This is the Django server. <Directory $POLICYKIT_REPO/policykit/policykit> <Files wsgi.py> Require all granted </Files> </Directory> WSGIDaemonProcess policykit python-home=$POLICYKIT_ENV python-path=$POLICYKIT_REPO/policykit WSGIProcessGroup policykit WSGIScriptAlias / $POLICYKIT_REPO/policykit/policykit/wsgi.py # .. REST ELIDED </VirtualHost> </IfModule>
Test your config with
apache2ctl configtest
. You should get a “Syntax OK” as a response.Enable your site:
# activate your config a2ensite /etc/apache2/sites-available/$SERVER_NAME.conf # disable the default config sudo a2dissite 000-default-le-ssl.conf
Get an SSL certificate and set it up to auto-renew using LetsEncrypt:
sudo apt install certbot python3-certbot-apache sudo certbot --apache
Add the certificates to your
$SERVER_NAME.conf
file:SSLCertificateFile /etc/letsencrypt/live/$SERVER_NAME/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/$SERVER_NAME/privkey.pem
Reload the config:
systemctl reload apache2
Give the Apache2 user access to the database directory (if using sqlite) and the logging directory (update paths as needed):
sudo chown -R www-data:www-data /var/log/django sudo chown -R www-data:www-data /var/databases/policykit
Load your site in the browser and navigate to
/login
. You should see a site titled “Django adminstration” with options to connect to Slack, Reddit, Discourse, and Discord. Before you can install PolicyKit into any of these platforms, you’ll need to set the necessary client IDs and client inprivate.py
. Follow the setup instructions for each integration in Integrations.
Check for errors at
/var/log/apache2/error.log
and/var/log/django/debug.log
(or whatever logging path you set in.env
).
Any time you update the code, you’ll need to run
systemctl reload apache2
to reload the server.
Set up Celery¶
PolicyKit uses Celery to run scheduled tasks.
Follow these instructions to run a celery daemon on your Ubuntu machine using systemd
.
For more information about configuration options, see the Celery Daemonization.
Create celery user¶
If you don’t already have a celery
user, create one:
sudo useradd celery -d /home/celery -b /bin/bash
Give the celery
user access to necessary pid and log folders:
sudo useradd celery -d /home/celery -b /bin/bash
sudo mkdir /var/log/celery
sudo chown -R celery:celery /var/log/celery
sudo chmod -R 755 /var/log/celery
sudo mkdir /var/run/celery
sudo chown -R celery:celery /var/run/celery
sudo chmod -R 755 /var/run/celery
The celery
user will also need write access to the Django log file and the database.
To give celery
access, create a group that contains both www-data
(the apache2 user) and celery
.
For example, if your Django logs are in /var/log/django
and your database is in /var/databases
:
sudo groupadd www-and-celery
sudo usermod -a -G www-and-celery celery
sudo usermod -a -G www-and-celery www-data
# give the group read-write access to logs
sudo chgrp -R www-and-celery /var/log/django
sudo chmod -R 775 /var/log/django
# give the group read-write access to database (if using sqlite)
sudo chgrp -R www-and-celery /var/databases
sudo chmod -R 775 /var/databases
Create Celery configuration files¶
Next, you’ll need to create three Celery configuration files for PolicyKit:
/etc/conf.d/celery
¶
CELERYD_NODES="w1"
# Absolute or relative path to the 'celery' command:
CELERY_BIN="$POLICYKIT_ENV/bin/celery"
# App instance to use
CELERY_APP="policykit"
# How to call manage.py
CELERYD_MULTI="multi"
# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"
# - %n will be replaced with the first part of the nodename.
# - %I will be replaced with the current child process index
# and is important when using the prefork pool to avoid race conditions.
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"
# you may wish to add these options for Celery Beat
CELERYBEAT_PID_FILE="/var/run/celery/beat.pid"
CELERYBEAT_LOG_FILE="/var/log/celery/beat.log"
/etc/systemd/system/celery.service
¶
[Unit]
Description=Celery Service
After=network.target
[Service]
Type=forking
User=celery
Group=celery
EnvironmentFile=/etc/conf.d/celery
WorkingDirectory=$POLICYKIT_REPO/policykit
ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} \
-A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
--logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \
--pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \
-A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
--logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
[Install]
WantedBy=multi-user.target
/etc/systemd/system/celerybeat.service
¶
[Unit]
Description=Celery Beat Service
After=network.target
[Service]
Type=simple
User=celery
Group=celery
EnvironmentFile=/etc/conf.d/celery
WorkingDirectory=$POLICYKIT_REPO/policykit
ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} \
beat --pidfile=${CELERYBEAT_PID_FILE} \
--logfile=${CELERYBEAT_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} \
--schedule=/var/run/celery/celerybeat-schedule'
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo service rabbitmq-server start
sudo systemctl start celery celerybeat
sudo systemctl status celery
sudo systemctl status celerybeat
Troubleshooting¶
celery -A policykit worker -l info --uid celery
celery -A policykit beat -l info --uid celery --schedule=/var/run/celery/celerybeat-schedule
If celerybeat experiences errors starting up, check the logs at /var/log/celery/beat.log
.
Interactive Django Shell¶
The interactive Django shell can be useful when developing and debugging PolicyKit.
Access the Django shell with python manage.py shell_plus
.
Some useful shell commands for development:
# List all communities
Community.objects.all()
# List CommunityPlatforms for a specific community
community = Community.objects.first()
CommunityPlatform.objects.filter(community=community)
# Get all pending proposals
Proposal.objects.filter(status="proposed")
# Manually run the policy checking task that is executed on a schedule by Celery
from policyengine.tasks import evaluate_pending_proposals
evaluate_pending_proposals()
###### Advanced Commands for debugging Metagov ######
# Access the Metagov Community model
from metagov.core.models import Community as MetagovCommunity
MetagovCommunity.objects.all()
MetagovCommunity.objects.get(slug=community.metagov_slug)
# Access the Metagov Plugin models (1:1 with CommunityPlatform)
Plugin.objects.all()
Slack.objects.all()
Plugin.objects.filter(community__slug=community.metagov_slug)
# Get pending Metagov GovernanceProcesses
GovernanceProcess.objects.filter(status='pending')
GovernanceProcess.objects.filter(plugin__community=metagov_community)
SlackEmojiVote.objects.filter(status='pending', plugin__community__slug="my-slug")
Set up Integrations¶
Before your instance of PolicyKit can be installed onto external platforms, you’ll need to go through setup steps for each integration that you want to support:
Slack¶
The Slack integration occurs through Metagov. Follow the setup instructions for the Metagov Slack Plugin to create a new Slack App to use with PolicyKit.
Discord¶
The Discord integration occurs through Metagov. Follow the setup instructions for the Metagov Discord Plugin to create a new Discord App to use with PolicyKit.
Discourse¶
There is no admin setup required for Discourse. Each Discourse community that installs PolicyKit needs to register the PolicyKit auth redirect separately.
Reddit¶
Create a new app at https://www.reddit.com/prefs/apps
Set the
REDDIT_CLIENT_SECRET
inprivate.py
.Reload apache2:
systemctl reload apache2
Developing the Metagov Gateway¶
If you’re making changes to the Metagov Gateway and want to test those changes in PolicyKit, you have two options:
Push your changes to a branch or fork, and update
requirements.txt
in PolicyKit to point to it:-e git+https://github.com/metagov/gateway.git@<your-dev-branch>#egg=metagov&subdirectory=metagov
Use pip “editable” installs to point to your local Metagov Gateway codebase:
pip install -e /path/to/gateway/repo/metagov