Refactor buildsystem to NPM/Gulp, port to BS4 & rewrite RIGS tests accordingly... (#412)

* Start to seperate versioning into its own app * Start reworking invoice things * Reduced overall font size a touch * Improvements to generic lists * Tweak some colours to be a bit less OTT I need to work out if I can seperate background and primary colours like BS3 did * Improvements to event table mobile * First pass at mobile-ising the generic list * Item table fixes * Fixed fullcalendar print css not included * Asset list table improvements * Tweak asset list to be more in line with other lists * Versioning template improvements //TODO Rather than have seperate asset templates, convert 'id' into a template variable * Tweak versioning templates to allow ID overrides Asset specific templates begone. Still need to bring back the ID formatting for the Rigboard. * Asset form fixes * Use the right autocompleter.js... * Breakout (most) user stuff to separate module The model remains in RIGS for now, as it's pretty painful to move... * Python Format/import opt * Test Refactor Part 1 - Shuffle things around * Fix migrations TODO - need to ensure moved models are *moved* rather than deleted and recreated! * Start on new tests * Initial work on event create test reimpl * Init other tests, more rigs test faffery * Desaturate theme colors even more Much closer to BS3 * Fix event item adding Bit too heavy handed with the deduplication there Arona * Initial refactor of event item testing * Upgrade bootstrap-select * Updated bootstrap-select for BS4 * Initial port of duplicate testing Needs the latter half rewriting once we have an EventDetail POM * Refactor date validation test So close to killing test_functional.EventTest! * Deduplication of testing code * pep8 * Fix some tests And some things that were actually borked * FIX: Prevent setting access time after start time Cherry pick of d274ea4606. Will close #405. * Refactor calendar tests * FIX: Don't show asset buttons/history for basic users * Really ought to get a pre-commit hook for pep8... * Fully replace test_functional * Dedupe generic search logic * Fix the remaining tests * Ensure submit button is scrolled to in tests * Fix asset creation test + actually verify its results * Make CI use latest (stable) chromedriver rather than some ancient one Since Travis uses the latest stable chrome, should always match. Bash oneliner \o/ * Of course | is part of YAML syntax, of course... Maybe this works. * Update python version Trying to get CI to match my local environment as much as possible... * Minor test futzing * Well that wasn't clever of me * That was even less clever of me * Revert to old submit wait behaviour * What about if I did this * Try disabling chrome cache * Added screenshot recording of test failures * Fixed RIGS tests not being run * Fixed Pep8 - I promise I'll make a pre-commit hook sometime! * Very initial work at togglable darktheme. Dammit @alexdaniel654 just when I had my scope creep kinda under control. It'll be v. nice to have though...! * More dark theme wangling * Fix some asset template things * FIX: CI Locale Issues * Fix sample command * Initial work at integrating the risk assessment #136. No clever database structure as yet... * FIX: Don't set every boolean input to radios * Different approach to RA linking * Move text definitions to somewhere more authoratitive * FIX: Undo breakage causing autopep8 o.O * Expand detail template * Use correct view for RA history * Initial work at coercing activity feed into showing RAs Also shows Asset/Supplier on the homepage feed. * Refactor activity feed template logic Yay for removing arbitrary if/else chains! * Initial work on caching activity feed Server side that is. Ref #162. * Start RA list template * Refactor RA creation stuff, again * Add H&S Details to Event Detail View * Display venue notes in event detail Notes are no use if nobody reads them. Not sure on this one. * Add ability to filter event archive by status Closes #168. * Fix lingering naive time * Use locmem cache in sqlite environments Otherwise the tests just lock up totally. Should close #162 * Update dependencies Mirrors/supersedes 0e67da82e2 * Add global ctrl/meta-enter shortcut for form submission Wants rewriting for better efficiency, but hey, it works! * Update dependencies * Fix for a situation that should be impossible * Fix navbar alignment * FEAT: Improve 'omni'search - Partialised template - Added to assets header - Added ability to search assets/suppliers - Improved selection logic - Have it display current query * Move closemodal into PyRIGS * Fix tests for search improvements * Dark mode colour improvements * Fix table colors for dry hires * further darktheme fixes * Remove the dark header from light theme * Fix reload loops when CSS/JS is changed * Move dark theme SCSS to separate file, fix inactive pagination styling * Genercise detail pages * Testing something re notes I wonder if I can make that global, rather than per-template... * Dark theme palette shenanigans I just can't decide * Match darktheme palette to forum darktheme palette Why reinvent the wheel. * Make supplier detail use the generic template * Disable mobile event table PoC for now * Remove the defaults from the RA fields + make them required * More RA fixes * Fixes to revisions for RAs * Add bootstrap 4 test page * Bunch of dark mode fixes from test page * Do not use Django 'required' for radio selects As this requires them to be True, whereas we just need to require that an option be entered. * Properly fixed popover darktheme * Fixed search for events * Style fixes to asset list * Start RA 'mark review' feature * Add reviewing to revision history, fix RA editing not working Also actually commit all the files, that helps * Fix Power MIC being lost on RA edit Why it is subtly different to the Event Update behaviour? Who knows * Invalidate RA review if it is edited after review * Start work on event checklist * Add a button for creating and instantly voiding invoices Handy dandy for when you have loads of cancelled events, like say, a pandemic * Mooooore status chips, mooore * Initial shenanigans on storing my overly fancy EC form * Proof of concept for JSON parsing/storage \o/ * Add new line functionality for vehicles/drivers Might it have been easier to create 'dummy' models like with EventItems? Probably... * Alter rig_count to not include un-checked-in dry hires * Insert a divider between still-out dry hires and actually upcoming events on rigboard * Initial work on new checklist handling. No more JSON! * Versioning module now does magic Automatic creation of views/urls for anything registered with reversion, with a small amount of hackage to preserve legacy stuff. (and the DAMNED asset IDs!) I would never get distracted... * Cleanup * Event checklist crew works Mostly - its not happy with timezones * Medium event power stuff done, barring worst case stuff * Misc fixes * Validation of power reqs * Worst case points on checklist * Templating improvements to RA/EC stuff * Do event table color logic at python level * Audit template fixes * Restrict versioning to one level of depth for speed Also fixed the template for nested changes * Event properties internal/authorised always return a explicit boolean rather than sometimes None * Use template filter for notes * Fix list templates TODO: Sensible place to define the 'expected answer' stuff. * Fix cable table template * Rethink rigboard color logic again Also revert some broken stuff * Test fixes * Modify auth test so it doesn't try and test for external authorisations Cause that's not a thing * Why does this work Bloody overzealous autoformatter... * Formatting... * Initial work on RA tests * Pages/start of tests for EventChecklists * Much better coverage of H&S things * Cleanup & Squash migrations * Fix wrong variable name in settings.py * Fix broken invoice list template * Add revision history to invoices/payments. Also patches previously introduced reversion permissions hole. Supersedes and closes #337. * Various misc fixes * Fix for my fix * Curse youuuuu pep8 * Invoice template improvements * Minor fixes * More tweaks * More fixes * Major improvements/fixes to authorisation templates * Add ability to mark event checklists as Large Event This just disables the checks to allow the rest of it to be filled out for large events, though I expect paper forms may still be used... * Remove database ID from generic list * Put power threshold values in a collapse * Use template filter for consistent removal of 'None links' Plus cleaner template markup! More HTML-in-Python tho, which always feels a bit CSS-in-JS * Tweak asset list markup * Begin to change add buttons success -> primary Also change search primary -> info to avoid clash * Begin to improve event checklist on mobile * Asset detail template improvements * Fix #326 (again) * Fix errors being squashed * Fix rigboard validation tests * Initial work on BS4 button templatetag Newfeatureitis strikes again * Allow multiple event checklists per event TODO: Status chip now needs rethinking * Minor event detail fixes * Fix tests * Rework button tag * Mobile fixes for search * Fix event checklist on mobile * Redo light theme palette * Switch rigboard new button to primary * Kill off excess whitespace on rigboard * Rigboard Timing display tweaks * Fix tests * Properly handle eventauthorisations in new versioning It's not great, not terrible... * Prevent creating duplicate revisions on event Potential fix for #322 - I couldn't reproduce even before this change... * Template improvements * Minor test fixes * Revert "Prevent creating duplicate revisions on event" Apparently it was too strong at preventing dupes... This reverts commit cce0ad0f9f. # Conflicts: # RIGS/models.py * Better approach to generic list templates + other deduplication * Also apply better approach to generic detail pages * One of these days I'll remember to test BEFORE pushing... * And now the same for generic forms * Display tick/cross rather than true/false in boolean version diffs * Upgrade dependencies * Fixes fixes fixes * Fix dependency hell Probably * Correct handling of spaces in paperwork filenames Also normalises display of Invoice IDs. Partial fix for #391. * Buggerit millennium hand and shrimp Knew I was gonna forget to fix the tests * FIX: Set duplicated event status to provisional Closes #398. Flip flop. Flip flop. * Update polyfill for datetime-local Bloody Firefox. We love to hate you. Proper CSS of the fill to come, SoonTM. Closes #391 * Curses! * Minor typo fixes * Initial pass at soop-consult confirmation screen for RAs * Fix migration * Make venue/date editable on EC For multi venue, multi day events Defaults to date and venue set on the event. Also made power MIC default to that set in RA * Clearer logic for RA inverted fields * (probably) fix tests * Give keyholders supplier edit perm * Generic list only displays edit button if user has perm * Same perm check for generic details * H&S Details takes up free space on non-internal events * Remove flash of content when loading new rig page * First pass at clearer display of asset list filters * Fix tests / default to headless tests (fingers crossed) * Fix autocompleter.js to properly disable edit links again * Move status color logic back to template Cause that somehow makes it work better?? * Display note icon on event detail page * Fix caching * Put rounded corners back where they belong * Remove lingering use of 'page-header' BS removed that style * More search and replace for BS changes Thought I'd got them all. Clearly not! * Remove enforced linebreak on status chips * Fix horizontal-ness on some forms * Remove animation on prefers-reduced-motion/low referesh rate devices Also normalises handling of asset list cable table & improves its use of space on large devices * Make version changes badges more readable * First pass at making the calendar less crap * Fix event table success logic Yay for copy paste fails >.> * Use borders rather than block colors for coloured tables under darktheme * First pass at porting calendar from FC V3 to V5 Two major versions and all they did was rename a bunch of names...TWICE. * Rework version name method to avoid blank names on eventchecklist vehicles/crew * Fix cable test * Made radio button focus much more obvious on dark theme * Implement Jerb's wording changes * Fix one test, break another... * Fix recent change stream list mutation issue * FIX: Do not naively cache event table Not that easy, it turns out. Duh. * FEAT: Implement #413 show associated assets on cable type detail pg Closes #413 * Allow H&S for non-events * Update emergency contact number * Improvements to profile detail page * Implement some of Jonny's suggested changes TODO: - Define event size at RA time, pass through to EC - Have later power questions be context dependent * Test fixes * Add space for power/rigging plans to be linked to RAs * Start move of event size logic to RA from Ec * Javascript required shenanigans for RA power * More moving of event size logic * Fixing tests for new logic etc * Why does this work Indeed, it may not * FIX: Stupid typo in versioning.py * Further minor fixes to versioning * Add icons to H&S menu items * Should fix calendar breaking in production * Small alignment fix in asset list * Squash migrations Co-authored-by: Matthew Smith <psyms13@nottingham.ac.uk>
2026-01-19 06:22:16 +00:00 · 2021-01-23 22:22:37 +00:00
parent 099a184f2e
commit 3414204209
336 changed files with 59425 additions and 33101 deletions
--- a/PyRIGS/decorators.py
+++ b/PyRIGS/decorators.py
@@ -8,7 +8,8 @@ from RIGS import models

 def get_oembed(login_url, request, oembed_view, kwargs):
    context = {}
-    context['oembed_url'] = "{0}://{1}{2}".format(request.scheme, request.META['HTTP_HOST'], reverse(oembed_view, kwargs=kwargs))
+    context['oembed_url'] = "{0}://{1}{2}".format(request.scheme, request.META['HTTP_HOST'],
+                                                  reverse(oembed_view, kwargs=kwargs))
    context['login_url'] = "{0}?{1}={2}".format(login_url, REDIRECT_FIELD_NAME, request.get_full_path())
    resp = render(request, 'login_redirect.html', context=context)
    return resp
@@ -28,9 +29,11 @@ def has_oembed(oembed_view, login_url=None):
                    return get_oembed(login_url, request, oembed_view, kwargs)
                else:
                    return HttpResponseRedirect('%s?%s=%s' % (login_url, REDIRECT_FIELD_NAME, request.get_full_path()))
+
        _checklogin.__doc__ = view_func.__doc__
        _checklogin.__dict__ = view_func.__dict__
        return _checklogin
+
    return _dec


@@ -60,9 +63,11 @@ def user_passes_test_with_403(test_func, login_url=None, oembed_view=None):
                resp = render(request, '403.html')
                resp.status_code = 403
                return resp
+
        _checklogin.__doc__ = view_func.__doc__
        _checklogin.__dict__ = view_func.__dict__
        return _checklogin
+
    return _dec


@@ -80,6 +85,7 @@ def api_key_required(function):
    Failed users will be given a 403 error.
    Should only be used for urls which include <api_pk> and <api_key> kwargs
    """
+
    def wrap(request, *args, **kwargs):

        userid = kwargs.get('api_pk')
@@ -101,6 +107,7 @@ def api_key_required(function):
        if user_object.api_key != key:
            return error_resp
        return function(request, *args, **kwargs)
+
    return wrap


@@ -108,11 +115,13 @@ def nottinghamtec_address_required(function):
    """
    Checks that the current user has an email address ending @nottinghamtec.co.uk
    """
+
    def wrap(request, *args, **kwargs):
        # Fail if current user's email address isn't @nottinghamtec.co.uk
        if not request.user.email.endswith('@nottinghamtec.co.uk'):
-            error_resp = render(request, 'RIGS/eventauthorisation_request_error.html')
+            error_resp = render(request, 'eventauthorisation_request_error.html')
            return error_resp

        return function(request, *args, **kwargs)
+
    return wrap
--- a/PyRIGS/formats/en/formats.py
+++ b/PyRIGS/formats/en/formats.py
@@ -1,5 +1,3 @@
-
-
 DATETIME_FORMAT = ('d/m/Y H:i')
 DATE_FORMAT = ('d/m/Y')
 TIME_FORMAT = ('H:i')
--- a/PyRIGS/forms.py
+++ b/PyRIGS/forms.py
--- a/PyRIGS/settings.py
+++ b/PyRIGS/settings.py
@@ -26,9 +26,10 @@ SECRET_KEY = os.environ.get('SECRET_KEY') if os.environ.get(
 # SECURITY WARNING: don't run with debug turned on in production!
 DEBUG = bool(int(os.environ.get('DEBUG'))) if os.environ.get('DEBUG') else True

-
 STAGING = bool(int(os.environ.get('STAGING'))) if os.environ.get('STAGING') else False

+CI = bool(int(os.environ.get('CI'))) if os.environ.get('CI') else False
+
 ALLOWED_HOSTS = ['pyrigs.nottinghamtec.co.uk', 'rigs.nottinghamtec.co.uk', 'pyrigs.herokuapp.com']

 if STAGING:
@@ -45,7 +46,8 @@ if not DEBUG:

 INTERNAL_IPS = ['127.0.0.1']

-ADMINS = [('Tom Price', 'tomtom5152@gmail.com'), ('IT Manager', 'it@nottinghamtec.co.uk'), ('Arona Jones', 'arona.jones@nottinghamtec.co.uk')]
+ADMINS = [('Tom Price', 'tomtom5152@gmail.com'), ('IT Manager', 'it@nottinghamtec.co.uk'),
+          ('Arona Jones', 'arona.jones@nottinghamtec.co.uk')]
 if DEBUG:
    ADMINS.append(('Testing Superuser', 'superuser@example.com'))

@@ -57,6 +59,9 @@ INSTALLED_APPS = (
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
+    'django.contrib.humanize',
+    'versioning',
+    'users',
    'RIGS',
    'assets',

@@ -147,6 +152,27 @@ LOGGING = {
    }
 }

+# Tests lock up SQLite otherwise
+if STAGING or CI:
+    CACHES = {
+        'default': {
+            'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'
+        }
+    }
+elif DEBUG:
+    CACHES = {
+        'default': {
+            'BACKEND': 'django.core.cache.backends.dummy.DummyCache'
+        }
+    }
+else:
+    CACHES = {
+        'default': {
+            'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
+            'LOCATION': 'cache_table',
+        }
+    }
+
 RAVEN_CONFIG = {
    'dsn': os.environ.get('RAVEN_DSN'),
    # If you are using git, you can also automatically configure the
@@ -164,8 +190,10 @@ LOGOUT_URL = '/user/logout/'
 ACCOUNT_ACTIVATION_DAYS = 7

 # reCAPTCHA settings
-RECAPTCHA_PUBLIC_KEY = os.environ.get('RECAPTCHA_PUBLIC_KEY', "6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI")  # If not set, use development key
-RECAPTCHA_PRIVATE_KEY = os.environ.get('RECAPTCHA_PRIVATE_KEY', "6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe")  # If not set, use development key
+RECAPTCHA_PUBLIC_KEY = os.environ.get('RECAPTCHA_PUBLIC_KEY',
+                                      "6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI")  # If not set, use development key
+RECAPTCHA_PRIVATE_KEY = os.environ.get('RECAPTCHA_PRIVATE_KEY',
+                                       "6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe")  # If not set, use development key
 NOCAPTCHA = True

 SILENCED_SYSTEM_CHECKS = ['captcha.recaptcha_test_key_error']
@@ -201,10 +229,10 @@ USE_L10N = True

 USE_TZ = True

+# Need to allow seconds as datetime-local input type spits out a time that has seconds
 DATETIME_INPUT_FORMATS = ('%Y-%m-%dT%H:%M', '%Y-%m-%dT%H:%M:%S')

 # Static files (CSS, JavaScript, Images)
-# https://docs.djangoproject.com/en/1.7/howto/static-files/
 STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
 STATIC_URL = '/static/'
 STATIC_ROOT = os.path.join(BASE_DIR, 'static/')
@@ -216,7 +244,7 @@ TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
-            os.path.join(BASE_DIR, 'templates'),
+            os.path.join(BASE_DIR, 'templates')
        ],
        'APP_DIRS': True,
        'OPTIONS': {
@@ -243,3 +271,6 @@ RISK_ASSESSMENT_URL = os.environ.get('RISK_ASSESSMENT_URL') if os.environ.get(
    'RISK_ASSESSMENT_URL') else "http://example.com"
 RISK_ASSESSMENT_SECRET = os.environ.get('RISK_ASSESSMENT_SECRET') if os.environ.get(
    'RISK_ASSESSMENT_SECRET') else secrets.token_hex(15)
+
+IMGUR_UPLOAD_CLIENT_ID = os.environ.get('IMGUR_UPLOAD_CLIENT_ID', '')
+IMGUR_UPLOAD_CLIENT_SECRET = os.environ.get('IMGUR_UPLOAD_CLIENT_SECRET', '')
--- a/PyRIGS/tests/base.py
+++ b/PyRIGS/tests/base.py
@@ -3,13 +3,31 @@ from selenium import webdriver
 from RIGS import models as rigsmodels
 from . import pages
 import os
+import pytz
+from datetime import date, time, datetime, timedelta
+from django.conf import settings
+import imgurpython
+import PyRIGS.settings
+import sys
+import pathlib
+import inspect
+
+
+def create_datetime(year, month, day, hour, min):
+    tz = pytz.timezone(settings.TIME_ZONE)
+    return tz.localize(datetime(year, month, day, hour, min)).astimezone(pytz.utc)


 def create_browser():
    options = webdriver.ChromeOptions()
    options.add_argument("--window-size=1920,1080")
+    # No caching, please and thank you
+    options.add_argument("--aggressive-cache-discard")
+    options.add_argument("--disk-cache-size=0")
+    # God Save The Queen
+    options.add_argument("--lang=en_GB")
+    options.add_argument("--headless")
    if os.environ.get('CI', False):
-        options.add_argument("--headless")
        options.add_argument("--no-sandbox")
    driver = webdriver.Chrome(options=options)
    return driver
@@ -34,3 +52,48 @@ class AutoLoginTest(BaseTest):
        self.profile.save()
        loginPage = pages.LoginPage(self.driver, self.live_server_url).open()
        loginPage.login("EventTest", "EventTestPassword")
+
+
+def screenshot_failure(func):
+    def wrapper_func(self, *args, **kwargs):
+        try:
+            func(self, *args, **kwargs)
+        except Exception as e:
+            screenshot_name = func.__module__ + "." + func.__qualname__
+            screenshot_file = "screenshots/" + func.__qualname__ + ".png"
+            if not pathlib.Path("screenshots").is_dir():
+                os.mkdir("screenshots")
+            self.driver.save_screenshot(screenshot_file)
+
+            if settings.IMGUR_UPLOAD_CLIENT_ID != "":
+                config = {
+                    'album': None,
+                    'name': screenshot_name,
+                    'title': screenshot_name,
+                    'description': ""
+                }
+                client = imgurpython.ImgurClient(settings.IMGUR_UPLOAD_CLIENT_ID, settings.IMGUR_UPLOAD_CLIENT_SECRET)
+                image = client.upload_from_path(screenshot_file, config=config)
+                print("Error in test {} is at url {}".format(screenshot_name, image['link']), file=sys.stderr)
+            else:
+                print("Error in test {} is at path {}".format(screenshot_name, screenshot_file), file=sys.stderr)
+            raise e
+    return wrapper_func
+
+
+def screenshot_failure_cls(cls):
+    for attr in cls.__dict__:
+        if callable(getattr(cls, attr)) and attr.startswith("test"):
+            setattr(cls, attr, screenshot_failure(getattr(cls, attr)))
+    return cls
+
+
+# Checks if animation is done
+class animation_is_finished():
+    def __call__(self, driver):
+        numberAnimating = driver.execute_script('return $(":animated").length')
+        finished = numberAnimating == 0
+        if finished:
+            import time
+            time.sleep(0.1)
+        return finished
--- a/PyRIGS/tests/pages.py
+++ b/PyRIGS/tests/pages.py
@@ -1,4 +1,5 @@
 from pypom import Page, Region
+from selenium.webdriver.common.action_chains import ActionChains
 from selenium.webdriver.common.by import By
 from selenium.webdriver import Chrome
 from selenium.common.exceptions import NoSuchElementException
@@ -30,14 +31,19 @@ class BasePage(Page):

 class FormPage(BasePage):
    _errors_selector = (By.CLASS_NAME, "alert-danger")
+    _submit_locator = (By.XPATH, "//button[@type='submit' and contains(., 'Save')]")

    def remove_all_required(self):
-        self.driver.execute_script("Array.from(document.getElementsByTagName(\"input\")).forEach(function (el, ind, arr) { el.removeAttribute(\"required\")});")
-        self.driver.execute_script("Array.from(document.getElementsByTagName(\"select\")).forEach(function (el, ind, arr) { el.removeAttribute(\"required\")});")
+        self.driver.execute_script(
+            "Array.from(document.getElementsByTagName(\"input\")).forEach(function (el, ind, arr) { el.removeAttribute(\"required\")});")
+        self.driver.execute_script(
+            "Array.from(document.getElementsByTagName(\"select\")).forEach(function (el, ind, arr) { el.removeAttribute(\"required\")});")

    def submit(self):
        previous_errors = self.errors
-        self.find_element(*self._submit_locator).click()
+        submit = self.find_element(*self._submit_locator)
+        ActionChains(self.driver).move_to_element(submit).perform()
+        submit.click()
        self.wait.until(lambda x: self.errors != previous_errors or self.success)

    @property
--- a/PyRIGS/tests/regions.py
+++ b/PyRIGS/tests/regions.py
@@ -1,9 +1,12 @@
 from pypom import Region
+from django.utils import timezone
 from selenium.webdriver.common.by import By
 from selenium.webdriver.support import expected_conditions
 from selenium.webdriver.remote.webelement import WebElement
 from selenium.webdriver.support.ui import WebDriverWait
 from selenium.webdriver.support.select import Select
+from selenium.webdriver.common.keys import Keys
+from selenium.common.exceptions import NoSuchElementException
 import datetime


@@ -16,11 +19,22 @@ def parse_bool_from_string(string):
    else:
        return False

+# 12-Hour vs 24-Hour Time. Affects widget display
+
+
+def get_time_format():
+    # Default
+    time_format = "%H:%M"
+    # If system is 12hr
+    if timezone.now().strftime("%p"):
+        time_format = "%I:%M %p"
+    return time_format
+

 class BootstrapSelectElement(Region):
    _main_button_locator = (By.CSS_SELECTOR, 'button.dropdown-toggle')
    _option_box_locator = (By.CSS_SELECTOR, 'ul.dropdown-menu')
-    _option_locator = (By.CSS_SELECTOR, 'ul.dropdown-menu.inner>li>a[role=option]')
+    _option_locator = (By.CSS_SELECTOR, 'ul.dropdown-menu.inner>li>a.dropdown-item')
    _select_all_locator = (By.CLASS_NAME, 'bs-select-all')
    _deselect_all_locator = (By.CLASS_NAME, 'bs-deselect-all')
    _search_locator = (By.CSS_SELECTOR, '.bs-searchbox>input')
@@ -32,12 +46,12 @@ class BootstrapSelectElement(Region):

    def toggle(self):
        original_state = self.is_open
-        return self.find_element(*self._main_button_locator).click()
        option_box = self.find_element(*self._option_box_locator)
-        if original_state:
-            self.wait.until(expected_conditions.invisibility_of_element_located(option_box))
+        if not original_state:
+            self.wait.until(expected_conditions.invisibility_of_element(option_box))
        else:
-            self.wait.until(expected_conditions.visibility_of_element_located(option_box))
+            self.wait.until(expected_conditions.visibility_of(option_box))
+        return self.find_element(*self._main_button_locator).click()

    def open(self):
        if not self.is_open:
@@ -55,6 +69,7 @@ class BootstrapSelectElement(Region):

    def search(self, query):
        search_box = self.find_element(*self._search_locator)
+        self.open()
        search_box.clear()
        search_box.send_keys(query)
        status_text = self.find_element(*self._status_locator)
@@ -112,6 +127,22 @@ class CheckBox(Region):
            self.toggle()


+class RadioSelect(Region):  # Currently only works for yes/no radio selects
+    def set_value(self, value):
+        if value:
+            value = "0"
+        else:
+            value = "1"
+        self.find_element(By.XPATH, "//label[@for='{}_{}']".format(self.root.get_attribute("id"), value)).click()
+
+    @property
+    def value(self):
+        try:
+            return parse_bool_from_string(self.find_element(By.CSS_SELECTOR, '.custom-control-input:checked').get_attribute("value").lower())
+        except NoSuchElementException:
+            return None
+
+
 class DatePicker(Region):
    @property
    def value(self):
@@ -122,6 +153,32 @@ class DatePicker(Region):
        self.root.send_keys(value.strftime("%d%m%Y"))


+class TimePicker(Region):
+    @property
+    def value(self):
+        return datetime.datetime.strptime(self.root.get_attribute("value"), get_time_format())
+
+    def set_value(self, value):
+        self.root.clear()
+        self.root.send_keys(value.strftime(get_time_format()))
+
+
+class DateTimePicker(Region):
+    @property
+    def value(self):
+        return datetime.datetime.strptime(self.root.get_attribute("value"), "%Y-%m-%d " + get_time_format())
+
+    def set_value(self, value):
+        self.root.clear()
+
+        date = value.date().strftime("%d%m%Y")
+        time = value.time().strftime(get_time_format())
+
+        self.root.send_keys(date)
+        self.root.send_keys(Keys.TAB)
+        self.root.send_keys(time)
+
+
 class SingleSelectPicker(Region):
    @property
    def value(self):
@@ -155,3 +212,39 @@ class ErrorPage(Region):
        for error in error_items:
            errors[error.field_name] = error.errors
        return errors
+
+
+class Modal(Region):
+    _submit_locator = (By.CSS_SELECTOR, '.btn-primary')
+    _header_selector = (By.TAG_NAME, 'h4')
+
+    form_items = {
+        'name': (TextBox, (By.ID, 'id_name'))
+    }
+
+    @property
+    def header(self):
+        return self.find_element(*self._header_selector).text
+
+    @property
+    def is_open(self):
+        return self.root.is_displayed()
+
+    def submit(self):
+        self.root.find_element(*self._submit_locator).click()
+
+    def __getattr__(self, name):
+        if name in self.form_items:
+            element = self.form_items[name]
+            form_element = element[0](self, self.find_element(*element[1]))
+            return form_element.value
+        else:
+            return super().__getattribute__(name)
+
+    def __setattr__(self, name, value):
+        if name in self.form_items:
+            element = self.form_items[name]
+            form_element = element[0](self, self.find_element(*element[1]))
+            form_element.set_value(value)
+        else:
+            self.__dict__[name] = value
--- a/PyRIGS/urls.py
+++ b/PyRIGS/urls.py
@@ -2,24 +2,36 @@ from django.urls import path
 from django.conf.urls import include, url
 from django.contrib import admin
 from django.contrib.staticfiles.urls import staticfiles_urlpatterns
+from django.contrib.auth.decorators import login_required
 from django.conf import settings
-from registration.backends.default.views import RegistrationView
+from django.views.decorators.clickjacking import xframe_options_exempt
+from django.contrib.auth.views import LoginView
+from django.views.generic import TemplateView
+from PyRIGS.decorators import permission_required_with_403
 import RIGS
-from RIGS import regbackend
+import users
+import versioning
+from PyRIGS import views

 urlpatterns = [
-    # Examples:
-    # url(r'^$', 'PyRIGS.views.home', name='home'),
-    # url(r'^blog/', include('blog.urls')),
+    path('', include('versioning.urls')),
+    path('', include('RIGS.urls')),
+    path('assets/', include('assets.urls')),

-    url(r'^', include('RIGS.urls')),
-    url('^assets/', include('assets.urls')),
-    url('^user/register/$', RegistrationView.as_view(form_class=RIGS.forms.ProfileRegistrationFormUniqueEmail),
-        name="registration_register"),
-    path('user/', include('django.contrib.auth.urls')),
-    path('user/', include('registration.backends.default.urls')),
+    path('', login_required(views.Index.as_view()), name='index'),

-    url(r'^admin/', admin.site.urls),
+    # API
+    path('api/<str:model>/', login_required(views.SecureAPIRequest.as_view()),
+         name="api_secure"),
+    path('api/<str:model>/<int:pk>/', login_required(views.SecureAPIRequest.as_view()),
+         name="api_secure"),
+
+    path('closemodal/', views.CloseModal.as_view(), name='closemodal'),
+    path('search_help/', views.SearchHelp.as_view(), name='search_help'),
+
+    path('', include('users.urls')),
+
+    path('admin/', admin.site.urls),
 ]

 if settings.DEBUG:
@@ -28,4 +40,5 @@ if settings.DEBUG:
    import debug_toolbar
    urlpatterns = [
        url(r'^__debug__/', include(debug_toolbar.urls)),
+        path('bootstrap/', TemplateView.as_view(template_name="bootstrap.html")),
    ] + urlpatterns
--- a/PyRIGS/views.py
+++ b/PyRIGS/views.py
@@ -0,0 +1,247 @@
+from django.core.exceptions import PermissionDenied
+from django.http.response import HttpResponseRedirect
+from django.http import HttpResponse
+from django.urls import reverse_lazy, reverse, NoReverseMatch
+from django.views import generic
+from django.contrib.auth.views import LoginView
+from django.db.models import Q
+from django.shortcuts import get_object_or_404
+from django.core import serializers
+from django.conf import settings
+import simplejson
+from django.contrib import messages
+import datetime
+import pytz
+import operator
+from registration.views import RegistrationView
+from django.views.decorators.csrf import csrf_exempt
+
+from RIGS import models, forms
+from assets import models as asset_models
+from functools import reduce
+
+from django.views.decorators.cache import never_cache, cache_page
+from django.utils.decorators import method_decorator
+
+
+# Displays the current rig count along with a few other bits and pieces
+class Index(generic.TemplateView):
+    template_name = 'index.html'
+
+    def get_context_data(self, **kwargs):
+        context = super(Index, self).get_context_data(**kwargs)
+        context['rig_count'] = models.Event.objects.rig_count()
+        return context
+
+
+class SecureAPIRequest(generic.View):
+    models = {
+        'venue': models.Venue,
+        'person': models.Person,
+        'organisation': models.Organisation,
+        'profile': models.Profile,
+        'event': models.Event,
+        'supplier': asset_models.Supplier
+    }
+
+    perms = {
+        'venue': 'RIGS.view_venue',
+        'person': 'RIGS.view_person',
+        'organisation': 'RIGS.view_organisation',
+        'profile': 'RIGS.view_profile',
+        'event': None,
+        'supplier': None
+    }
+
+    '''
+    Validate the request is allowed based on user permissions.
+    Raises 403 if denied.
+    Potential to add API key validation at a later date.
+    '''
+
+    def __validate__(self, request, key, perm):
+        if request.user.is_active:
+            if request.user.is_superuser or perm is None:
+                return True
+            elif request.user.has_perm(perm):
+                return True
+        raise PermissionDenied()
+
+    def get(self, request, model, pk=None, param=None):
+        # Request permission validation things
+        key = request.GET.get('apikey', None)
+        perm = self.perms[model]
+        self.__validate__(request, key, perm)
+
+        # Response format where applicable
+        format = request.GET.get('format', 'json')
+        fields = request.GET.get('fields', None)
+        if fields:
+            fields = fields.split(",")
+
+        # Supply data for one record
+        if pk:
+            object = get_object_or_404(self.models[model], pk=pk)
+            data = serializers.serialize(format, [object], fields=fields)
+            return HttpResponse(data, content_type="application/" + format)
+
+        # Supply data for autocomplete ajax request in json form
+        term = request.GET.get('q', None)
+        if term:
+            if fields is None:  # Default to just name
+                fields = ['name']
+
+            # Build a list of Q objects for use later
+            queries = []
+            for part in term.split(" "):
+                qs = []
+                for field in fields:
+                    q = Q(**{field + "__icontains": part})
+                    qs.append(q)
+                queries.append(reduce(operator.or_, qs))
+
+            # Build the data response list
+            results = []
+            query = reduce(operator.and_, queries)
+            objects = self.models[model].objects.filter(query)
+            for o in objects:
+                data = {
+                    'pk': o.pk,
+                    'value': o.pk,
+                    'text': o.name,
+                }
+                try:  # See if there is a valid update URL
+                    data['update'] = reverse("%s_update" % model, kwargs={'pk': o.pk})
+                except NoReverseMatch:
+                    pass
+                results.append(data)
+
+            # return a data response
+            json = simplejson.dumps(results)
+            return HttpResponse(json, content_type="application/json")  # Always json
+
+        start = request.GET.get('start', None)
+        end = request.GET.get('end', None)
+
+        if model == "event" and start and end:
+            # Probably a calendar request
+            start_datetime = datetime.datetime.strptime(start, "%Y-%m-%dT%H:%M:%S")
+            end_datetime = datetime.datetime.strptime(end, "%Y-%m-%dT%H:%M:%S")
+
+            objects = self.models[model].objects.events_in_bounds(start_datetime, end_datetime)
+
+            results = []
+            for item in objects:
+                data = {
+                    'pk': item.pk,
+                    'title': item.name,
+                    'is_rig': item.is_rig,
+                    'status': str(item.get_status_display()),
+                    'earliest': item.earliest_time.isoformat(),
+                    'latest': item.latest_time.isoformat(),
+                    'url': str(item.get_absolute_url())
+                }
+
+                results.append(data)
+            json = simplejson.dumps(results)
+            return HttpResponse(json, content_type="application/json")  # Always json
+
+        return HttpResponse(model)
+
+
+class ModalURLMixin:
+    def get_close_url(self, update, detail):
+        if self.request.is_ajax():
+            url = reverse_lazy('closemodal')
+            update_url = str(reverse_lazy(update, kwargs={'pk': self.object.pk}))
+            messages.info(self.request, "modalobject=" + serializers.serialize("json", [self.object]))
+            messages.info(self.request, "modalobject[0]['update_url']='" + update_url + "'")
+        else:
+            url = reverse_lazy(detail, kwargs={
+                'pk': self.object.pk,
+            })
+        return url
+
+
+class GenericListView(generic.ListView):
+    template_name = 'generic_list.html'
+    paginate_by = 20
+
+    def get_context_data(self, **kwargs):
+        context = super(GenericListView, self).get_context_data(**kwargs)
+        context['page_title'] = self.model.__name__ + "s"
+        if self.request.is_ajax():
+            context['override'] = "base_ajax.html"
+        return context
+
+    def get_queryset(self):
+        q = self.request.GET.get('q', "")
+
+        filter = Q(name__icontains=q) | Q(email__icontains=q) | Q(address__icontains=q) | Q(notes__icontains=q) | Q(
+            phone__startswith=q) | Q(phone__endswith=q)
+
+        # try and parse an int
+        try:
+            val = int(q)
+            filter = filter | Q(pk=val)
+        except:  # noqa
+            # not an integer
+            pass
+
+        object_list = self.model.objects.filter(filter)
+
+        orderBy = self.request.GET.get('orderBy', "name")
+        if orderBy != "":
+            object_list = object_list.order_by(orderBy)
+        return object_list
+
+
+class GenericDetailView(generic.DetailView):
+    template_name = "generic_detail.html"
+
+    def get_context_data(self, **kwargs):
+        context = super(GenericDetailView, self).get_context_data(**kwargs)
+        context['page_title'] = "{} | {}".format(self.model.__name__, self.object.name)
+        if self.request.is_ajax():
+            context['override'] = "base_ajax.html"
+        return context
+
+
+class GenericUpdateView(generic.UpdateView):
+    template_name = "generic_form.html"
+
+    def get_context_data(self, **kwargs):
+        context = super(GenericUpdateView, self).get_context_data(**kwargs)
+        context['page_title'] = "Edit {}".format(self.model.__name__)
+        if self.request.is_ajax():
+            context['override'] = "base_ajax.html"
+        return context
+
+
+class GenericCreateView(generic.CreateView):
+    template_name = "generic_form.html"
+
+    def get_context_data(self, **kwargs):
+        context = super(GenericCreateView, self).get_context_data(**kwargs)
+        context['page_title'] = "Create {}".format(self.model.__name__)
+        if self.request.is_ajax():
+            context['override'] = "base_ajax.html"
+        return context
+
+
+class SearchHelp(generic.TemplateView):
+    template_name = 'search_help.html'
+
+
+"""
+Called from a modal window (e.g. when an item is submitted to an event/invoice).
+May optionally also include some javascript in a success message to cause a load of
+the new information onto the page.
+"""
+
+
+class CloseModal(generic.TemplateView):
+    template_name = 'closemodal.html'
+
+    def get_context_data(self, **kwargs):
+        return {'messages': messages.get_messages(self.request)}
--- a/PyRIGS/wsgi.py
+++ b/PyRIGS/wsgi.py
@@ -7,6 +7,7 @@ For more information on this file, see
 https://docs.djangoproject.com/en/1.7/howto/deployment/wsgi/
 """
 import os
+
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "PyRIGS.settings")

 from django.core.wsgi import get_wsgi_application  # noqa