To Do R5.5

R 5.5

Roadmap
dev repo [bitbucket.org]
ChangeLog
Team Sessions


Main Focus: basic HTML5 support, address security related features

  1. HTTP Strict Transport Security (HSTS)
  2. Content-Security-Policy 2
  3. Cookies

Excursus

  1. Strict-Transport-Security HTTP Response Header
    • Instructs the browser to always request a domain using the HTTPS protocol instead of HTTP.
  2. Content-Security-Policy (CSP)
    • HTTP Response header, allows server to control how resources are loaded.

1. Security Headers

  1. foster http_security_headers() implementation
    1. https://www.owasp.org/index.ph[..]_useful_HTTP_headers
    2. https://www.owasp.org/index.php/Content_Security_Policy
    3. http://www.w3.org/TR/CSP2/
    4. https://w3c.github.io/webappsec-csp/
  2. abandon vendor prefixes -> Content-Security-Policy (Suggestion: avoid prefixed implementations)
    1. Content-Security-Policy:
       default-src 'self';
       script-src 'self' 'unsafe-eval' ajax.googleapis.com google-analytics.com;
       style-src 'self' ajax.googleapis.com;
       connect-src 'self' https://api.myapp.com realtime.myapp.com:8080;
       media-src 'self' youtube.com;
       object-src 'self' youtube.com;
       child-src 'self' youtube.com embed.ly
    2. Content-Security-Policy:
       default-src 'self';
       script-src 'self' 'unsafe-inline';
       style-src 'self' 'unsafe-inline';
       img-src *;
      • replace inline scripts

1.1. Replace inline JavaScript

https://en.wikipedia.org/wiki/Unobtrusive_JavaScript



Replace inline JavaScript


oncontextmenu
onmousedown
onload
onclick
onerror
onmouseover
onmouseout

  1. <script>var dbclick = "page";</script>

https://stackoverflow.com/ques[..]d-vs-document-onload



//'Getting' data-attributes using dataset 
var showhandler = document.getElementById('showhandler');
var dbclick = showhandler.dataset.dbclick1; // dbclick = 'page';

2. SafeHTML / HTMLPurifier

  1. implement optional support for HTMLPurifier
    1. http://htmlpurifier.org/download
    2. http://repo.or.cz/htmlpurifier.git/shortlog/
  2. http://www.bioinformatics.org/[..]_utilities/htmLawed/

3. HTML5 Migration

  1. update form attributes http://www.w3.org/TR/html5/forms.html#the-input-element
    1. placeholder="info@example.com"
    2. placeholder="E-Mail-Adresse eingeben"

4. Features

  1. normalize links to other language versions of a page
    • add table: lang_link[page_id, lang, target_id]

5. RC3

open issues (add)

  1. add delimiter before page handler (_properties)
    1. min_href()
    2. router.conf
    3. abandon standard_handlers (?)
  2. guide to build templates -> parent file -> template
    1. how do templates work
    2. $tpl->naming
    3. how to process conditions
  3. save failed posts for reuse after forced logout
  4. unwrap_link($tag) gives back wrong result -> ignores base_url
    • SELECT DISTINCT a.page_id, a.tag, a.title, a.hits FROM doc_page a, doc_page_link l INNER JOIN doc_page b ON (l.from_page_id = b.page_id) INNER JOIN doc_page c ON (l.to_page_id = c.page_id) WHERE a.tag <> 'Doc/Doc/Deutsch' AND a.tag = c.tag AND INSTR(b.tag, 'Doc/Doc/Deutsch') = 1 AND INSTR(c.tag, 'Doc/Doc/Deutsch') = 1 ORDER BY a.hits DESC LIMIT 10
    • first Doc is part of base_url, e.g.
      • {{mostpopular for="/Doc/Deutsch" max=10 nomark=1 title=1}}
      • {{mostpopular for="English" max=6 nomark=1 title=1}}
  5. debug_admin_only is broken
  6. disable cache for message after forced logout: $http->no_cache();
  7. invalidate sql and page cache (with common function) (?), which is also checking against config settings
  8. add also footer after hard return, GUI consistency
  9. upload of file without extention -> broken
  10. allow only common reasonable extensions for upload!!!
  11. add notifications for new page -> Admin, Moderator, parent page owner
  12. add notification for new attachments
    1. -> to page: watchers, Admin, Moderator
    2. -> global: Admin, Moderator
  13. add default max value in actions: news, blog, files, etc. -> use list_count as default
  14. empty body_r after page rename/relocation -> page needs rerendering
  15. installer: CSS routing using subdomain?
  16. installer: rewrite_mode option ! – dysfunctional
  17. https://wackowiki.org/doc/Bugs%2FWackoWikiStart ->
    Not Found
    
    The requested URL /doc/Bugs/WackoWikiStart was not found on this server.
  18. redirect hashid url?
  19. delete / reset pages with missing language, e.g. 'mo', upgrade or AP routine
  20. allow multi logins (on/off)
    1. add multi login warning: 'Jemand hat sich bereits an diesem Konto angemeldet'
  21. add access throttling feature
    • limit the number of page requests by a single IP address within a given time interval
  22. useful messages for regular users after logout
  23. load translation is loaded before resource for theme lang is set for theme_per_page, FIXED same issue for user theme
  24. add array for 'default' AND 'user' menu so both can used independently (create/edit menu sets)
  25. common header can't be reused for none template themes -> fix?
  26. rewrite_mode setting in AP is pointless if it is overwritten in Setting class
  27. wacko.all.php settings in themes were ignored! Fix?
    • <?php
      // tabs theme options =========
      	$this->db->revisions_hide_cancel = 1;
      	$this->db->footer_inside = 0;
      // ============================
      $theme_translation = [
      	'EditIcon' => '<img src="' . $this->db->theme_url . 'icon/edit.png" alt="Edit included page" />',
      	'' => '',
      ];
      ?>

6. Fix

  1. invalidate / purge only a sub set of the SQL cache (?), do we always need to purge the entire cache?
  2. check formatting of log() function -> html / wacko formatting
  3. file links were tracked only after a second rendering -> ?
  4. use common list count setting per user + use it as default in lists for paging
  5. check for avoidable SQL roundtrip queries
    1. e.g. translit: href() -> mini_href() -> slim_url() -> translit() ($this->config['multilanguage'] == true) -> load_page()
    2. fix additional round trips gaining page_id in combination with has_access() and link() function (tag <-> page_id)
      1. options: use object cache or pass variable page_id via link() function:
        • // cache page_id for for has_access validation in link function
        • $this->page_id_cache[$page['tag']] = $page['page_id'];
      2. part 1: http://wackowiki.hg.sourceforg[..]dev/rev/f5f2295a85b9
    3. add object for extending and array with default pages (accessible via theme) to cache_link() function
      1. -> ensure pre caching to avoid single lookup per intralink in DB on each page call
  6. link() -> default: $anchor_link – should only active inside page_body (?)
    • additional check if its better to prefix the id="doc.deutsch.konfiguration"
      • to avoid unintentional mix with CSS settings
    • set anchor id only where needed, minimizes also size of attributes
  7. add option help to action to show all parameters in a info box 
    1. echo ''
  8. <#<kbd>F1</kbd>#> – add css class for kbd tag 
  9. broken list in tree action if levels changes not in order – e.g. depth 1.2 -> depth 2.4
    1. show missing levels
  10. add function to replace random isset($_GET|$_POST) ? .. : null
    1. filter_input # gets a specific external variable by name and optionally filters it 
  11. search?phrase="sourceforge.net" -> paging fails with "term to search"
  12. bug: news action takes all subpages – is this desired?
  13. improve search (open since ages), add some measures to improve relevance (time, size, user, filter, ...) and provide more and better meta data for search results
  14. add options to show/hide page related categories at the page bottom
    • themes may overwrite these settings via $this->config['footer_tags'] = OFF
    • allways ON as default for posts in the forum cluster
    • do we need an additional option for the user?
  15. get translation
    1. put lang-strings for action and handlers into separate dynamically loadable lang-files
    2. cache
  16. audit comments, moderation handler
  17. better localization of "users" cluster
  18. replace p tag in toc action -> avoid wrong p in p 
  19. should we allow page names like chicken.egg, might conflict with other settings like tikiwiki formatter option
  20. review concept handling files per page file?get= -> performance, time, resources, necessity, alternatives
  21. Form in preview breaks Edit / Preview form -> produces nested form -> filter?
  22. revisit access right settings for forum posts and menu access
    1. the menu won't show the page properties icon -> annoying
  23. syntax
    1. table header
  24. implement rating hack (but without mandatory JS)
    1. https://www.youtube.com/watch?v=orPVEAipz2A
  25. add graphviz formatter to /community/formatter
  26. image action
    1. resize, cache
    2. using library
    3. store thumbnails in extra folder
      1. under files/ or _cache/ [..] thumbnails/
      2. global / per_page
  27. gallery action sharing functions with image action
  28. add a License / Standard Terms feature
    • page / file
  29. add unique log message key to filter events (messages may differ)
  30. use deleted field to mark deleted pages, comments, files
    • basics implemented for page and files
    • open: rollback/restore procedure and handling of final deletion
    • check how we do this for files alone and/or with related page (matrix)
    • WHERE clause from COUNT(*) queries
  31. IPv6
    • get_ip
  32. add meta name= date and last-modified ?
  33. disable registration in default config / disable installer via ?setting? by default
  34. CSS: display: table; for layout-box class (?)
  35. disable global upload for users
    1. only local
    2. only for cluster
  36. TODO: wacko class -> function validate_reserved_words( $data )
  37. add regex for this->config['users_page']/[*]/
    • Yet the engine does not validate the namespace for the user cluster, so that nobody can create a page under /User except his own [UserName]
    • Then we can disallow random pages for the first level in the users cluster except the own [UserName].
    • The register action creates this page usually for the user.

log DEFAULT CURRENT_TIMESTAMP

  • ALTER TABLE `wacko_log` CHANGE `log_time` `log_time` TIMESTAMP ON UPDATE CURRENT_TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ;

7. Notice digest

  • store events notices and compile digest for user [user|moderator|admin]
    • new user
    • comments
    • files
    • changes

8. Installer

  1. installer: add missing form label fields
  2. colliding page names with multiple languages with the same name -> insert_page()
    1. creates only the first match, other page and menu creations will fail
    2. Solution: (A) UTF-8, (B) different page names, e.g. index, index
  3. legacy upgrade: SQL strict mode and missing default values -> HOTFIX: set default values manually via phpMyAdmin
    1. see table structure and select all rows with Default -> None
    2. chose change at the bottom and change Default -> None to As defined: and save
    3. repeat this for all related tables if necessary

9. Handler

  1. add paging to revisions
  2. review transliteration of file names in upload handler: white space, '-', '_'
  3. clone entire cluster is only available foe Admins atm., it should also available for ...
  4. improve global upload settings
    1. allow groups
    2. set individual rights (only images, quota, etc. for a user, group)
  5. send page as email (like print)
  6. show: add option 'Flag as Spam/Inappropriate'
  7. diff: show time, revision, user and change note for side A and B 
  8. show: Delayed Indexing delay_index
    • <meta name=“robots” content=“noindex,nofollow” />
  9. upload: check if the MIME type of the uploaded file matches the file extension https://www.acunetix.com/websitesecurity/upload-forms-threat/
  10. upload: add form field to chose another file name (?)
  11. upload: add accept attribute depending on config settings https://www.w3.org/TR/html5/fo[..]ml#attr-input-accept
  12. upload: send a notify mail on upload
  13. unify form label style, see filemeta, properties, account handler
    • make label secondary ->

10. Formatter

  1. replace icq formatters with common chat formatter
  2. (/Users/WikiAdmin UserSpace | WikiAdmin)) -fails on |
  3. Image links to other sites
    • The automatically added symbol for an outerlink is misplaced here, e.g.:
      • ((http://example.com/ https://www.example.com/media/img/logo.png))
  4. Wacko is spamming BRs, in between everything
  5. Wacko is putting P.auto around DIV elements = Plain Simple Bullshit -> bugs:375
    1. should not set paragraphs in cases like
    2. <p class="auto" id="p96596-1"><!--notypo--><pre class="code">
      <p class="auto" id="p21312-1"><!--notypo--><div class="layout-box">
    3. leads to invalid html tag nesting
  6. Error: Bad value 4 for attribute type on element ol.
    1. see $new_indent_type in wackoformatter -> error prone
      • 1. hallo
            5. sould not take the number but 1, same for i, I, a, A
    2. Block elements inside inline elements
    3. http://www.w3.org/TR/html5/gro[..]nt.html#attr-ol-type
  7. even if wiki_links were turned off (disable_wikilinks) the formatter should try to form links for in intralinks with at least one slash (?) like
    • /Dev/Release/R50/ReleaseNotes#h1433-7
  8. place holder
    <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR4nGP6zwAAAgcBApocMXEAAAAASUVORK5CYII="/>
    1. div.image {
      	width:            100px;
      	height:           100px;
      	background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR4nGP6zwAAAgcBApocMXEAAAAASUVORK5CYII=');
      }
  9. allow case insensitive matching of file links, e.g. File:image.jpg
  10. add option to add rel= noreferrer and nofollow to external links
  11. breaks quote
    <[http://www.example.com]>
  12. ((image.jpg)) shows images from image/ folder ?
  13. display of correct headings inside wrapper %%(wacko wrapper="shade") with toc is broken
  14. https://de.wikipedia.org/wiki/Gehörnte_Mauerbiene
  15. <[ ]> eats blank line in quote, undesired
  16. broken nested quote
    • <[block
      <[nested quote]>
      quote
      ]>

11. Cache

12. Admin Panel

  1. add Check for Updates button in Admin panel: /Download/VersionCheck
  2. Synchronizing data: update comment count for page if out of sync
  3. user management
    • deactivate / delete inactive users
      • criteria
      • actions
  4. add module to filter, moderate and manage pages, comments, (files)
    1. see modules for content like pages
  5. recovery mode: CSS and images won't load

13. Database

  1. https://www.digitalocean.com/c[..]ur-mysql-5-7-upgrade
  2. /Dev/Guidelines/SQL/SQLmodes

13.1. Check for SQL STRICT mode violations

  1. Using GROUP BY and selecting an ambiguous column
  2. Inserting the non standard zero date into a datetime column
  3. Inserting a 20 character string into a 10 character column
  4. Division by zero
  5. Inserting a negative value into an unsigned column

so far

  1. #1406 – Data too long for column 'description' at row 1
    1. set HTML maxlength="DB_FIELDSIZE" for all (VAR)CHAR form field
      • suggested (JS hint – might differ in some cases – smaller, e.g. meta description 160, meta title 60) + database field size (mandatory enforcement)
      • JS hint: You have <strong>60</strong> characters left
    2. set PHP length check before passing to INPUT / UPDATE
  2. #1055 – 'dev.g.group_name' isn't in GROUP BY

SELECT
`DIGEST_TEXT` AS `query`,
`SCHEMA_NAME` AS `db`,
`COUNT_STAR` AS `exec_count`,
`SUM_ERRORS` AS `errors`,
(ifnull((`SUM_ERRORS` / nullif(`COUNT_STAR`,0)),0) * 100) AS `error_pct`,
`SUM_WARNINGS` AS `warnings`,
(ifnull((`SUM_WARNINGS` / nullif(`COUNT_STAR`,0)),0) * 100) AS `warning_pct`,
`FIRST_SEEN` AS `first_seen`,
`LAST_SEEN` AS `last_seen`,
`DIGEST` AS `digest`
FROM
 performance_schema.events_statements_summary_by_digest
WHERE
((`SUM_ERRORS` > 0) OR (`SUM_WARNINGS` > 0))
ORDER BY
 `SUM_ERRORS` DESC,
 `SUM_WARNINGS` DESC;

14. WikiEdit

15. Translations

-> /Dev/Release/R55/Translations


add

  1. "\n\n"
    "Click on the following link to view the page:\n\n"
    • function notify_watcher

improve

  1. 'EmailRegistered' => 'You\'ve successfully registered at %1.'."\n".'Your username: %2'."\n\n".'If you want to receive notifications, you must click on the link below or copy it to a web browser.'."\n".'%3'."\n\n".'Please return to the Wiki and login with your new username and password.'."\n\n\n".'If you did not request any registration, ignore this message and nothing will happen.'."\n\n".'Do not reply to this message'."\n\n".'',

16. Usability issues

  1. Not indicating an active form field
    • e.g.
      textarea:focus {
          border: 1px solid red; }
    • You can use the ‘:focus’ selector on lots of elements, but it’s super handy when used on inputs and textareas to indicate that the field is active. Add CSS styling such as a highlighted border, or a subtle change to the background color.
  2. forms with checkboxes and options in lists
    • e.g. category handler or users in admin panel
    • assignment of form buttons
  3. usage of new page handler
    • seen in many fresh installs, users adding sub pages to HomePage/subpage
      • this is possible but is it really desired and understood, should we filter out system pages as pre-provided cluster in the /new [page] handler?
  4. make name spaces for users and groups more intuitive accessible
    1. users/nickname/userspace/..
    2. groups/usability/groupspace/..
  5. add ability to easily create groups and to add group members
    1. suggestions?
  6. GUI inconsistencies
    1. handler
    2. actions
    3. message boxes

Readings

  1. https://www.nngroup.com/articles/low-contrast/
  2. https://backchannel.com/how-th[..]eadable-a781ddc711b6

17. Libs

  1. update PHPMailer to v6.0
  2. update Hashids to v2.0

18. Extentions

19. Refactoring

  1. start replacing magic numbers at least with true and false where possible
  2. erode the mountain of technical debt
  3. https://www.owasp.org/index.php/Logging_Cheat_Sheet

20. Staging Area

  1. blog action (will replace news action)
  2. snippet action
  3. forum, topics action
  4. Admin Panel
    1. refactor -> antipatterns
    2. bad behaviour module
    3. list and check each module, assign status
  5. moderate handler -> still quite a mess
  6. poll actions -> still quite a mess

21. Ideas

  1. spam / badword handling -> bad_words($text) function
    1. https://en.wikipedia.org/wiki/Wordfilter
    2. https://stackoverflow.com/ques[..]ood-profanity-filter
    3. What you need is a good way for users to flag inappropriate content and a mechanism to deal with it swiftly. One way is to automatically hide/remove content if it's been flagged more than X times.
  2. HTML5 media action: {{media type="[audio|video|flash]" source=http://.... [width=000] [height=000] [...some other options...]}}
  3. test PHPThumb alternatives
    1. https://github.com/mosbth/cimage
  4. Composer
  5. https://github.com/openpgpjs/openpgpjs

22. Themes

  1. fix unintended use of mixed fonts
  2. new mobile ready theme / layout -> basic: http://holdirbootstrap.de/examples/theme/
  3. https://github.com/KDE/breeze-icons
  4. https://developers.google.com/web/fundamentals/

Flexbox vs Grid

  1. Flexbox: content dictates layout
  2. Grid: container dictates layout (to some extent)

Flexbox is great, it just isn't the best thing for overall page layouts.
Flexbox and grid play well together, and are a huge step forward from the float & table hacks they replace. The sooner we can use them both in production, the better.

default theme

  • change blockquote
    • box-shadow: 0 0 6px rgba(0, 0, 0, 0.5);
  • add option to hide $this->config['site_name'] in theme header, e.g.: WackoWiki: To Do R5.5 -> To Do R5.5
  • min-height: 200px for .article, #page

22.1. SVG

Icon setting: add viewBox="0 0 16 16" AND height="16" width="16"


explain

23. Documentation

  1. add documentation for Admin Panel
    1. backup and restore module
  2. Update / foster Core Documentation for
    1. Deutsch
    2. English
    3. Français
    4. Русский
  3. forum, topic action
  4. blog action -> Blogging with WackoWiki
  5. where and when you should use relative or absolute addressing (include action, files, actions with page parameter)
  6. how you use the include function (pages, comments)
  7. release notes
    1. add a New & Noteworthy section / sub page to raise visibility of 'hidden' features
  8. moderate handler
  9. changed form functions and form token validation
  10. message functions and usage
  11. add sub page to config documentation about CSP 
  12. Video Tutorials
  13. update screen shots

24. Feedback

  1. themes
  2. migration and encoding issues
  3. frequent annoying issues
  4. unsolved questions
  5. add rfc section in dev cluster
  6. add activity log 
  7. tag activities in communication: evaluation / RFC / testing / concepts / knowledge / etc. to make it easier to map what a certain topic of discussion focuses on, e.g. evaluation of new theme icons, implementation of HTMLPurifier, addressing of show stoppers for upcoming beta release

25. Test

  1. Tools
    1. https://developers.google.com/speed/pagespeed/insights/
    2. http://validator.w3.org/nu/
    3. https://validator.w3.org/feed/
    4. WAVE – Online accessibility validator
    5. http://jshint.com/
    6. https://www.google.com/webmast[..]ols/mobile-friendly/
  2. https://secure.php.net/manual/[..]n70.incompatible.php
  3. https://github.com/mozilla/readability – test with firefox reading mode
  4. /Forum/Discussion/UserPasswordReset -> solution paths

25.1. Debug

  1. XAMPP + Win 7: SessionFileStore: inaccessible directory "/tmp"
    1. define('CACHE_SESSION_DIR',				'_cache/sessions');
      #define('CACHE_SESSION_DIR',				'/tmp');
    2. Warning: file_put_contents(_cache/templates/theme:default:appearance:templates:header.tpl): failed to open stream: Invalid argument in C:\xampp\htdocs\wacko_fresh\wacko\class\templatest.php on line 135
      
      Warning: chmod(): No such file or directory in C:\xampp\htdocs\wacko_fresh\wacko\class\templatest.php on line 136
    3. Windows will reject the filename as it is because of the colons. Remove those and you should be fine.
  2. Content Security Policy: Die Einstellungen der Seite haben das Laden einer Ressource auf self blockiert ("script-src http://localhost 'unsafe-inline'").
    • call to eval() or related function blocked by CSP: autocomplete.js (Line 253)
  3. #1139 – Got error 'this version of PCRE is compiled without UTF support at offset 0' from regexp
    • Error
      
      Static analysis:
      
      8 errors were found during analysis.
      
          Unrecognized keyword. (near "REGEXP" at position 209)
          Unexpected token. (near "'^/Blog/.+/.+/.+$'" at position 216)
          Unrecognized keyword. (near "AND" at position 235)
          Unexpected token. (near "p" at position 239)
          Unexpected token. (near "." at position 240)
          Unexpected token. (near "deleted" at position 241)
          Unexpected token. (near "<>" at position 249)
          Unexpected token. (near "'1'" at position 252)
      
      SQL query: Documentation
      
      SELECT p.page_id, p.owner_id, p.user_id, p.tag, p.title, p.created, p.comments, u.user_name AS owner FROM wacko_page p INNER JOIN wacko_user u ON (p.owner_id = u.user_id) WHERE p.comment_on_id = '0' AND p.tag REGEXP '^/Blog/.+/.+/.+$' AND p.deleted <> '1' ORDER BY p.created DESC LIMIT 0, 10
      
      MySQL said: Documentation
      #1139 - Got error 'this version of PCRE is compiled without UTF support at offset 0' from regexp
    • https://stackoverflow.com/ques[..]sing-regexp-in-mysql

Read comment (1 comment)