Files
PyRIGS/versioning/versioning.py
Arona Jones 3414204209 Refactor buildsystem to NPM/Gulp, port to BS4 & rewrite RIGS tests accordingly... (#412)
* Start to seperate versioning into its own app

* Start reworking invoice things

* Reduced overall font size a touch

* Improvements to generic lists

* Tweak some colours to be a bit less OTT

I need to work out if I can seperate background and primary colours like BS3 did

* Improvements to event table mobile

* First pass at mobile-ising the generic list

* Item table fixes

* Fixed fullcalendar print css not included

* Asset list table improvements

* Tweak asset list to be more in line with other lists

* Versioning template improvements

//TODO Rather than have seperate asset templates, convert 'id' into a template variable

* Tweak versioning templates to allow ID overrides

Asset specific templates begone. Still need to bring back the ID formatting for the Rigboard.

* Asset form fixes

* Use the right autocompleter.js...

* Breakout (most) user stuff to separate module

The model remains in RIGS for now, as it's pretty painful to move...

* Python Format/import opt

* Test Refactor Part 1 - Shuffle things around

* Fix migrations

TODO - need to ensure moved models are *moved* rather than deleted and recreated!

* Start on new tests

* Initial work on event create test reimpl

* Init other tests, more rigs test faffery

* Desaturate theme colors even more

Much closer to BS3

* Fix event item adding

Bit too heavy handed with the deduplication there Arona

* Initial refactor of event item testing

* Upgrade bootstrap-select

* Updated bootstrap-select for BS4

* Initial port of duplicate testing

Needs the latter half rewriting once we have an EventDetail POM

* Refactor date validation test

So close to killing test_functional.EventTest!

* Deduplication of testing code

* pep8

* Fix some tests

And some things that were actually borked

* FIX: Prevent setting access time after start time 

Cherry pick of d274ea4606. Will close #405.

* Refactor calendar tests

* FIX: Don't show asset buttons/history for basic users

* Really ought to get a pre-commit hook for pep8...

* Fully replace test_functional

* Dedupe generic search logic

* Fix the remaining tests

* Ensure submit button is scrolled to in tests

* Fix asset creation test + actually verify its results

* Make CI use latest (stable) chromedriver rather than some ancient one

Since Travis uses the latest stable chrome, should always match. Bash oneliner \o/

* Of course | is part of YAML syntax, of course...

Maybe this works.

* Update python version

Trying to get CI to match my local environment as much as possible...

* Minor test futzing

* Well that wasn't clever of me

* That was even less clever of me

* Revert to old submit wait behaviour

* What about if I did this

* Try disabling chrome cache

* Added screenshot recording of test failures

* Fixed RIGS tests not being run

* Fixed Pep8 - I promise I'll make a pre-commit hook sometime!

* Very initial work at togglable darktheme.

Dammit @alexdaniel654 just when I had my scope creep kinda under control. It'll be v. nice to have though...!

* More dark theme wangling

* Fix some asset template things

* FIX: CI Locale Issues

* Fix sample command

* Initial work at integrating the risk assessment

#136. No clever database structure as yet...

* FIX: Don't set every boolean input to radios

* Different approach to RA linking

* Move text definitions to somewhere more authoratitive

* FIX: Undo breakage causing autopep8

o.O

* Expand detail template

* Use correct view for RA history

* Initial work at coercing activity feed into showing RAs

Also shows Asset/Supplier on the homepage feed.

* Refactor activity feed template logic

Yay for removing arbitrary if/else chains!

* Initial work on caching activity feed

Server side that is. Ref #162.

* Start RA list template

* Refactor RA creation stuff, again

* Add H&S Details to Event Detail View

* Display venue notes in event detail

Notes are no use if nobody reads them. Not sure on this one.

* Add ability to filter event archive by status

Closes #168.

* Fix lingering naive time

* Use locmem cache in sqlite environments

Otherwise the tests just lock up totally. Should close #162

* Update dependencies

Mirrors/supersedes 0e67da82e2

* Add global ctrl/meta-enter shortcut for form submission

Wants rewriting for better efficiency, but hey, it works!

* Update dependencies

* Fix for a situation that should be impossible

* Fix navbar alignment

* FEAT: Improve 'omni'search

- Partialised template
- Added to assets header
- Added ability to search assets/suppliers
- Improved selection logic
- Have it display current query

* Move closemodal into PyRIGS

* Fix tests for search improvements

* Dark mode colour improvements

* Fix table colors for dry hires

* further darktheme fixes

* Remove the dark header from light theme

* Fix reload loops when CSS/JS is changed

* Move dark theme SCSS to separate file, fix inactive pagination styling

* Genercise detail pages

* Testing something re notes

I wonder if I can make that global, rather than per-template...

* Dark theme palette shenanigans

I just can't decide

* Match darktheme palette to forum darktheme palette

Why reinvent the wheel.

* Make supplier detail use the generic template

* Disable mobile event table PoC for now

* Remove the defaults from the RA fields + make them required

* More RA fixes

* Fixes to revisions for RAs

* Add bootstrap 4 test page

* Bunch of dark mode fixes from test page

* Do not use Django 'required' for radio selects

As this requires them to be True, whereas we just need to require that an option be entered.

* Properly fixed popover darktheme

* Fixed search for events

* Style fixes to asset list

* Start RA 'mark review' feature

* Add reviewing to revision history, fix RA editing not working

Also actually commit all the files, that helps

* Fix Power MIC being lost on RA edit

Why it is subtly different to the Event Update behaviour? Who knows

* Invalidate RA review if it is edited after review

* Start work on event checklist

* Add a button for creating and instantly voiding invoices

Handy dandy for when you have loads of cancelled events, like say, a pandemic

* Mooooore status chips, mooore

* Initial shenanigans on storing my overly fancy EC form

* Proof of concept for JSON parsing/storage

\o/

* Add new line functionality for vehicles/drivers

Might it have been easier to create 'dummy' models like with EventItems? Probably...

* Alter rig_count to not include un-checked-in dry hires

* Insert a divider between still-out dry hires and actually upcoming events on rigboard

* Initial work on new checklist handling. No more JSON!

* Versioning module now does magic

Automatic creation of views/urls for anything registered with reversion, with a small amount of hackage to preserve legacy stuff. (and the DAMNED asset IDs!) I would never get distracted...

* Cleanup

* Event checklist crew works

Mostly - its not happy with timezones

* Medium event power stuff done, barring worst case stuff

* Misc fixes

* Validation of power reqs

* Worst case points on checklist

* Templating improvements to RA/EC stuff

* Do event table color logic at python level

* Audit template fixes

* Restrict versioning to one level of depth for speed

Also fixed the template for nested changes

* Event properties internal/authorised always return a explicit boolean rather than sometimes None

* Use template filter for notes

* Fix list templates

TODO: Sensible place to define the 'expected answer' stuff.

* Fix cable table template

* Rethink rigboard color logic again

Also revert some broken stuff

* Test fixes

* Modify auth test so it doesn't try and test for external authorisations

Cause that's not a thing

* Why does this work

Bloody overzealous autoformatter...

* Formatting...

* Initial work on RA tests

* Pages/start of tests for EventChecklists

* Much better coverage of H&S things

* Cleanup & Squash migrations

* Fix wrong variable name in settings.py

* Fix broken invoice list template

* Add revision history to invoices/payments.

Also patches previously introduced reversion permissions hole.

Supersedes and closes #337.

* Various misc fixes

* Fix for my fix

* Curse youuuuu pep8

* Invoice template improvements

* Minor fixes

* More tweaks

* More fixes

* Major improvements/fixes to authorisation templates

* Add ability to mark event checklists as Large Event

This just disables the checks to allow the rest of it to be filled out for large events, though I expect paper forms may still be used...

* Remove database ID from generic list

* Put power threshold values in a collapse

* Use template filter for consistent removal of 'None links'

Plus cleaner template markup! More HTML-in-Python tho, which always feels a bit CSS-in-JS

* Tweak asset list markup

* Begin to change add buttons success -> primary

Also change search primary -> info to avoid clash

* Begin to improve event checklist on mobile

* Asset detail template improvements

* Fix #326 (again)

* Fix errors being squashed

* Fix rigboard validation tests

* Initial work on BS4 button templatetag

Newfeatureitis strikes again

* Allow multiple event checklists per event

TODO: Status chip now needs rethinking

* Minor event detail fixes

* Fix tests

* Rework button tag

* Mobile fixes for search

* Fix event checklist on mobile

* Redo light theme palette

* Switch rigboard new button to primary

* Kill off excess whitespace on rigboard

* Rigboard Timing display tweaks

* Fix tests

* Properly handle eventauthorisations in new versioning

It's not great, not terrible...

* Prevent creating duplicate revisions on event

Potential fix for #322 - I couldn't reproduce even before this change...

* Template improvements

* Minor test fixes

* Revert "Prevent creating duplicate revisions on event"

Apparently it was too strong at preventing dupes...

This reverts commit cce0ad0f9f.

# Conflicts:
#	RIGS/models.py

* Better approach to generic list templates + other deduplication

* Also apply better approach to generic detail pages

* One of these days I'll remember to test BEFORE pushing...

* And now the same for generic forms

* Display tick/cross rather than true/false in boolean version diffs

* Upgrade dependencies

* Fixes fixes fixes

* Fix dependency hell

Probably

* Correct handling of spaces in paperwork filenames

Also normalises display of Invoice IDs. Partial fix for #391.

* Buggerit millennium hand and shrimp

Knew I was gonna forget to fix the tests

* FIX: Set duplicated event status to provisional

Closes #398.

Flip flop. Flip flop.

* Update polyfill for datetime-local

Bloody Firefox. We love to hate you. Proper CSS of the fill to come, SoonTM.

Closes #391

* Curses!

* Minor typo fixes

* Initial pass at soop-consult confirmation screen for RAs

* Fix migration

* Make venue/date editable on EC

For multi venue, multi day events

Defaults to date and venue set on the event. Also made power MIC default to that set in RA

* Clearer logic for RA inverted fields

* (probably) fix tests

* Give keyholders supplier edit perm

* Generic list only displays edit button if user has perm

* Same perm check for generic details

* H&S Details takes up free space on non-internal events

* Remove flash of content when loading new rig page

* First pass at clearer display of asset list filters

* Fix tests / default to headless tests

(fingers crossed)

* Fix autocompleter.js to properly disable edit links again

* Move status color logic back to template

Cause that somehow makes it work better??

* Display note icon on event detail page

* Fix caching

* Put rounded corners back where they belong

* Remove lingering use of 'page-header'

BS removed that style

* More search and replace for BS changes

Thought I'd got them all. Clearly not!

* Remove enforced linebreak on status chips

* Fix horizontal-ness on some forms

* Remove animation on prefers-reduced-motion/low referesh rate devices

Also normalises handling of asset list cable table & improves its use of space on large devices

* Make version changes badges more readable

* First pass at making the calendar less crap

* Fix event table success logic

Yay for copy paste fails >.>

* Use borders rather than block colors for coloured tables under darktheme

* First pass at porting calendar from FC V3 to V5

Two major versions and all they did was rename a bunch of names...TWICE.

* Rework version name method to avoid blank names on eventchecklist vehicles/crew

* Fix cable test

* Made radio button focus much more obvious on dark theme

* Implement Jerb's wording changes

* Fix one test, break another...

* Fix recent change stream list mutation issue

* FIX: Do not naively cache event table

Not that easy, it turns out. Duh.

* FEAT: Implement #413 show associated assets on cable type detail pg

Closes #413

* Allow H&S for non-events

* Update emergency contact number

* Improvements to profile detail page

* Implement some of Jonny's suggested changes

TODO:
- Define event size at RA time, pass through to EC
- Have later power questions be context dependent

* Test fixes

* Add space for power/rigging plans to be linked to RAs

* Start move of event size logic to RA from Ec

* Javascript required shenanigans for RA power

* More moving of event size logic

* Fixing tests for new logic etc

* Why does this work

Indeed, it may not

* FIX: Stupid typo in versioning.py

* Further minor fixes to versioning

* Add icons to H&S menu items

* Should fix calendar breaking in production

* Small alignment fix in asset list

* Squash migrations

Co-authored-by: Matthew Smith <psyms13@nottingham.ac.uk>
2021-01-23 22:22:37 +00:00

211 lines
7.3 KiB
Python

import datetime
import logging
from diff_match_patch import diff_match_patch
from django.contrib.contenttypes.models import ContentType
from django.core.exceptions import ObjectDoesNotExist
from django.db.models import EmailField, IntegerField, TextField, CharField, BooleanField
from django.shortcuts import get_object_or_404
from django.utils.functional import cached_property
from django.views import generic
from reversion.models import Version, VersionQuerySet
from RIGS import models
from assets import models as asset_models
from django.apps import apps
from reversion import revisions as reversion
logger = logging.getLogger('tec.pyrigs')
class FieldComparison(object):
def __init__(self, field=None, old=None, new=None):
self.field = field
self._old = old
self._new = new
def display_value(self, value):
if (isinstance(self.field, IntegerField) or isinstance(self.field, CharField)) and self.field.choices is not None and len(self.field.choices) > 0:
choice = [x[1] for x in self.field.choices if x[0] == value]
# TODO This defensive piece should not be necessary?
if len(choice) > 0:
return choice[0]
if isinstance(self.field, BooleanField):
if value:
return "&#10003;"
else:
return "&#10007"
return value
@property
def old(self):
return self.display_value(self._old)
@property
def new(self):
return self.display_value(self._new)
@property
def long(self):
if isinstance(self.field, EmailField):
return True
return False
@property
def linebreaks(self):
if isinstance(self.field, TextField):
return True
return False
@property
def diff(self):
oldText = str(self.display_value(self._old)) or ""
newText = str(self.display_value(self._new)) or ""
dmp = diff_match_patch()
diffs = dmp.diff_main(oldText, newText)
dmp.diff_cleanupSemantic(diffs)
outputDiffs = []
for (op, data) in diffs:
if op == dmp.DIFF_INSERT:
outputDiffs.append({'type': 'insert', 'text': data})
elif op == dmp.DIFF_DELETE:
outputDiffs.append({'type': 'delete', 'text': data})
elif op == dmp.DIFF_EQUAL:
outputDiffs.append({'type': 'equal', 'text': data})
return outputDiffs
class ModelComparison(object):
def __init__(self, old=None, new=None, version=None, follow=False, excluded_keys=[]):
# recieves two objects of the same model, and compares them. Returns an array of FieldCompare objects
try:
self.fields = old._meta.get_fields()
except AttributeError:
self.fields = new._meta.get_fields()
self.old = old
self.new = new
self.excluded_keys = excluded_keys
self.version = version
self.follow = follow
@cached_property
def revision(self):
return self.version.revision
@cached_property
def field_changes(self):
changes = []
for field in self.fields:
field_name = field.name
if field_name not in self.excluded_keys: # if we're excluding this field, skip over it
try:
oldValue = getattr(self.old, field_name, None)
except ObjectDoesNotExist:
oldValue = None
try:
newValue = getattr(self.new, field_name, None)
except ObjectDoesNotExist:
newValue = None
bothBlank = (not oldValue) and (not newValue)
if oldValue != newValue and not bothBlank:
comparison = FieldComparison(field, oldValue, newValue)
changes.append(comparison)
return changes
@cached_property
def fields_changed(self):
return len(self.field_changes) > 0
@cached_property
def item_changes(self):
if self.follow and self.version.object is not None:
item_type = ContentType.objects.get_for_model(self.version.object)
old_item_versions = self.version.parent.revision.version_set.exclude(content_type=item_type)
new_item_versions = self.version.revision.version_set.exclude(content_type=item_type).exclude(content_type=ContentType.objects.get_for_model(models.EventAuthorisation))
comparisonParams = {'excluded_keys': ['id', 'event', 'order', 'checklist']}
# Build some dicts of what we have
item_dict = {} # build a list of items, key is the item_pk
for version in old_item_versions: # put all the old versions in a list
compare = ModelComparison(old=version._object_version.object, **comparisonParams)
item_dict[version.object_id] = compare
for version in new_item_versions: # go through the new versions
try:
compare = item_dict[version.object_id] # see if there's a matching old version
compare.new = version._object_version.object # then add the new version to the dictionary
except KeyError: # there's no matching old version, so add this item to the dictionary by itself
compare = ModelComparison(new=version._object_version.object, **comparisonParams)
item_dict[version.object_id] = compare # update the dictionary with the changes
changes = []
for (_, compare) in list(item_dict.items()):
if compare.fields_changed:
changes.append(compare)
return changes
@cached_property
def name(self):
obj = self.new if self.new else self.old
if(hasattr(obj, 'activity_feed_string')):
return obj.activity_feed_string
else:
return str(obj)
@cached_property
def items_changed(self):
return self.item_changes is not None and len(self.item_changes) > 0
@cached_property
def anything_changed(self):
return self.fields_changed or self.items_changed
class RIGSVersionManager(VersionQuerySet):
def get_for_multiple_models(self, model_array):
content_types = []
for model in model_array:
content_types.append(ContentType.objects.get_for_model(model))
return self.filter(content_type__in=content_types).select_related("revision").order_by(
"-revision__date_created")
class RIGSVersion(Version):
class Meta:
proxy = True
objects = RIGSVersionManager.as_manager()
# Gets the most recent previous version
@cached_property
def parent(self):
thisId = self.object_id
versions = RIGSVersion.objects.get_for_object_reference(self.content_type.model_class(), thisId).select_related(
"revision", "revision__user").all()
try:
previousVersion = versions.filter(revision_id__lt=self.revision_id).latest('revision__date_created')
except ObjectDoesNotExist:
return False
return previousVersion
@cached_property
def changes(self):
return ModelComparison(
version=self,
new=self._object_version.object,
old=self.parent._object_version.object if self.parent else None,
follow=True
)