ToDo R5.5


Please pick the issue you can solve.

R 5.5

Roadmap
dev repo [bitbucket.org]
ChangeLog
Team Sessions


Main Focus: basic HTML5 support, address security related features

  1. HTTP Strict Transport Security (HSTS)
  2. Content-Security-Policy
  3. Cookies, CSRF sectoken
  4. https://www.owasp.org/index.ph[...]tication_Cheat_Sheet

1. PHP

1.1. PHP 7.4

  1. https://secure.php.net/manual/en/migration74.php

2. MariaDB / MySQL type casting

https://dev.mysql.com/doc/refm[...]type-conversion.html
https://stackoverflow.com/ques[...]d-backticks-in-mysql


whOOt https://dev.mysql.com/doc/refm[...]ent-programming.html


3. Features

  1. normalize links to other language versions of a page
    • add table: lang_link[page_id, lang, target_id]
  2. add debug option to send error log into separate file + rotate logs
  3. add IP block to ban bad actors, bots
  4. add timing method to prevent more bots from creating new accounts

4. M14

open issues (add)

  1. relative linking seems not working for ((../ back to parent page)), it links to root level (?!)
  2. wacko.all.php ($wacko_all_resource[]) is not available, when page_lang != user_lang, why?
    • related issue: added LicenseIds[] to wacko.all.php, guess what...
  3. add delimiter before page handler (_properties)
    1. min_href()
    2. router.conf
    3. abandon standard_handlers (?)
  4. Guide to build templates
  5. Permalink to page vs page version
  6. CSS does not get routed in RECOVERY_MODE
  7. save failed $_POST data for reuse after forced logout or autologin
  8. relative time
    1. global / user setting
    2. where to use relative time and where not 
    3. different schemes
  9. invalidate SQL and page cache (with common function) (?), which is also checking against config settings
  10. add also footer after hard return, GUI consistency
  11. how to check access privileges for a group, e.g. has_access() for $ -> moderate handler : locking
  12. upload of file without extension -> broken
  13. allow only common reasonable extensions for upload
    1. blacklist MIME types -> implement
    2. blacklist extension -> update
    3. white list
  14. add notification for 
    1. new page -> Admin, Moderator, parent page owner
    2. new attachments
      1. -> to page: watchers, Admin, Moderator
      2. -> global: Admin, Moderator
  15. add default max value in actions: news, blog, files, etc. -> use list_count as default
  16. empty body_r after page rename/relocation -> page needs rerendering
  17. installer: rewrite_mode option ! – dysfunctional
  18. redirect hashid url?
  19. delete / reset pages with missing language, e.g. 'mo', upgrade or AP routine
  20. allow multi logins (on/off)
    1. add multi login warning: 'Jemand hat sich bereits an diesem Konto angemeldet'
    2. This account is currently being used in 1 other location at this IP ().
  21. add access throttling feature
    • limit the number of page requests by a single IP address within a given time interval
  22. load translation is loaded before resource for theme lang is set for theme_per_page, FIXED same issue for user theme
  23. add array for 'default' AND 'user' menu so both can used independently (create/edit menu sets)
  24. common header can't be reused for none template themes -> fix?
  25. rewrite_mode setting in AP is pointless if it is overwritten in Setting class
  26. wacko.all.php settings in themes were ignored! Fix? 
    •  <?php
// tabs theme options =========
	$this->db->revisions_hide_cancel = 1;
	$this->db->footer_inside = 0;
// ============================
$theme_translation = [
	'EditIcon' => '<img src="' . $this->db->theme_url . 'icon/edit.png" alt="Edit included page" />',
	'' => '',
];
?>

5. Fix

  1. invalidate / purge only a sub set of the SQL cache (?), do we always need to purge the entire cache?
  2. check formatting of log() function -> html / wacko formatting
  3. use common list count setting per user + use it as default in lists for paging
  4. check for avoidable SQL roundtrip queries
    1. e.g. translit: href() -> mini_href() -> slim_url() -> translit() ($this->config['multilanguage'] == true) -> load_page()
    2. fix additional round trips gaining page_id in combination with has_access() and link() function (tag <-> page_id)
      1. options: use object cache or pass variable page_id via link() function:
        • // cache page_id for for has_access validation in link function
        • $this->page_id_cache[$page['tag']] = $page['page_id'];
      2. part 1: revision:f5f2295a85b9
    3. add object for extending and array with default pages (accessible via theme) to preload_link() function
      1. -> ensure pre caching to avoid single lookup per intralink in DB on each page call
      2. -> parse also page path for bread crumbs in page_link table
  5. link() -> default: $anchor_link – should only active inside page_body (?)
    • additional check if its better to prefix the id="doc.deutsch.konfiguration"
      • to avoid unintentional mix with CSS settings
    • set anchor id only where needed, minimizes also size of attributes
  6. add option help to action to show all parameters in a info box 
    1. echo 'show all available parameters with description'
  7. broken list in tree action if levels changes not in order – e.g. depth 1.2 -> depth 2.4
    1. show missing levels
  8. search?phrase="sourceforge.net" -> paging fails with "term to search"
  9. bug: news action takes all subpages – is this desired?
  10. improve search (open since ages), add some measures to improve relevance (time, size, user, filter, ...) and provide more and better meta data for search results
  11. add options to show/hide page related categories at the page bottom
    • themes may overwrite these settings via $this->config['footer_tags'] = OFF
    • allways ON as default for posts in the forum cluster
    • do we need an additional option for the user?
  12. get translation
    1. put lang-strings for action and handlers into separate dynamically loadable lang-files
    2. cache
  13. audit comments, moderation handler
  14. replace p tag in toc action -> avoid wrong p in p 
  15. should we allow page names like chicken.egg, might conflict with other settings like tikiwiki formatter option
  16. Form in preview breaks Edit / Preview form -> produces nested form -> filter?
  17. revisit access right settings for forum posts and menu access
    1. the menu won't show the page properties icon -> annoying
  18. syntax
    1. table header
  19. implement rating hack (but without mandatory JS)
    1. https://www.youtube.com/watch?v=orPVEAipz2A
  20. add graphviz formatter to /community/formatter
  21. add unique log message key to filter events (messages may differ)
  22. use deleted field to mark deleted pages, comments, files
    • basics implemented for page and files
    • open: rollback/restore procedure and handling of final deletion
  23. disable global upload for users
    1. only local
    2. only for cluster
  24. add regex for this->config['users_page']/[*]/
    • Yet the engine does not validate the namespace for the user cluster, so that nobody can create a page under /User except his own [UserName]
    • Then we can disallow random pages for the first level in the users cluster except the own [UserName].
    • The register action creates this page usually for the user.

6. Notifications

  1. email body and subject message encoding is pure shit in multi-language mode
    1. _t() function gives back html entities
  2. Notifications

6.1. Notice digest

  • store events notices and compile digest for user [user|moderator|admin]
    • new user
    • comments
    • files
    • changes

7. Installer

  1. colliding page names with multiple languages with the same name -> insert_page()
    1. creates only the first match, other page and menu creations will fail
    2. Solution: (A) UTF-8, (B) different page names, e.g. index, index
  2. legacy upgrade: SQL strict mode and missing default values -> HOTFIX: set default values manually via phpMyAdmin
    1. see table structure and select all rows with Default -> None
    2. chose change at the bottom and change Default -> None to As defined: and save
    3. repeat this for all related tables if necessary
  3. define('CACHE_SESSION_DIR', '/tmp'); is defined in constant.php, and currently not set via the installer
    1. use ini_get('session.save_path') as indicator, BUT we do not use the PHP build in session! -> write value, its a nuisance that the user currently has to do this on his own
  4. offer to create the recovery_password within the install process
  5. use dbal also for installer: $db->sql_query($sql)

8. Handler

  1. it sets occasionally the description text in page table to 1, WHY what is the error source causing this
  2. disallow white space in tags: new and moderate -> strip
3. 
preg_replace('/\s+/', '', $string);

  1. review transliteration of file names in upload handler: white space, '-', '_'
  2. clone entire cluster is only available foe Admins atm., it should also available for ...
  3. improve global upload settings
    1. allow groups
    2. set individual rights (only images, quota, etc. for a user, group)
  4. send page as email (like print)
  5. show: add option 'Flag as Spam/Inappropriate'
  6. diff: show time, revision, user and change note for side A and B 
  7. show: Delayed Indexing delay_index
    • <meta name=“robots” content=“noindex,nofollow”>
  8. see upload subpage
    1. upload: check if the MIME type of the uploaded file matches the file extension
      1. https://www.owasp.org/index.ph[...]stricted_File_Upload
      2. https://www.acunetix.com/websitesecurity/upload-forms-threat/
      3. https://secure.php.net/manual/[...]n.exif-read-data.php
    2. upload: add form field to chose another file name (?)
    3. upload: add accept attribute depending on config settings https://www.w3.org/TR/html5/fo[...]ml#attr-input-accept
    4. upload: send a notify mail on upload
  9. unify form label style, see filemeta, properties, account handler
    • make label secondary ->
  10. add meta handler namespace ['page', 'account', 'file', 'service']
    • This is the simplest way to standardize document locations and for the language-independent single instances of service pages, like login. Next step is the separated cluster for those pages, linked with prefix, for example, ((service:login)).
    • this can be easily done with the new URI router
    • handler/account/
  11. file: apply access restrictions for global files if Wiki is closed -> $
    1. add and enforce global Wiki mode, minimum access rights
    2. route global files only for registered users

9. Action

  1. template toc and tree

10. Formatter

  1. Search Highlighter
  2. a legit empty body_toc results in a repeated UPDATE of body_r on every page view -> see show handler FIXED
  3. (/Users/WikiAdmin UserSpace | WikiAdmin)) -fails on |
  4. <# #> adds <!--notypo--> on first and <!--/notypo--> on second appearance of double quote like class="" 
    1. <#<div class="" style="background:transparent; border:.1em solid #F66; border-left:1em solid #F66; box-sizing:border-box; margin:.5em 0; overflow:hidden; padding:.5em; text-align:left; width:auto;">Unter den Btrfs-spezifischen Anpassungen (1, 2) waren einige, die Latenz- und Stabilitäts-Probleme beseitigen, die bei knapp werdendem Speicherplatz auftreten können.</div>#>
2
<#<div class="" style="background:transparent; border:.1em solid #fcfce9; border-left:1em solid #fcfce9; box-sizing:border-box; margin:.5em 0; overflow:hidden; padding:.5em; text-align:left; width:auto;">Unter den Btrfs-spezifischen Anpassungen (1, 2) waren einige, die Latenz- und Stabilitäts-Probleme beseitigen, die bei knapp werdendem Speicherplatz auftreten können.</div>#>
    2. <!--notypo--><div class="<!--notypo--> style="background:transparent; border:.1em solid #F66; border-left:1em solid #F66; box-sizing:border-box; margin:.5em 0; overflow:hidden; padding:.5em; text-align:left; width:auto;">Unter den Btrfs-spezifischen Anpassungen (1, 2) waren einige, die Latenz- und Stabilitäts-Probleme beseitigen, die bei knapp werdendem Speicherplatz auftreten können.</div>#><br />2<br /><#<div class=<!--/notypo-->" style="background:transparent; border:.1em solid #fcfce9; border-left:1em solid #fcfce9; box-sizing:border-box; margin:.5em 0; overflow:hidden; padding:.5em; text-align:left; width:auto;">Unter den Btrfs-spezifischen Anpassungen (1, 2) waren einige, die Latenz- und Stabilitäts-Probleme beseitigen, die bei knapp werdendem Speicherplatz auftreten können.</div><!--/notypo--><br />
  5. Wacko is spamming BRs, in between everything
  6. add option to hide protected links
    • You must login to see this link. Register now, if you have no user account yet.
  7. Error: Bad value 4 for attribute type on element ol.
    1. see $new_indent_type in wackoformatter -> error prone 
      •   1. hallo
    5. should not take the number but 1, same for i, I, a, A
    2. Block elements inside inline elements
    3. http://www.w3.org/TR/html5/gro[...]nt.html#attr-ol-type
  8. even if wiki_links were turned off (disable_wikilinks) the formatter should try to form links for in intralinks with at least one slash (?) like
    • /Dev/Release/R5.0/ReleaseNotes#h1433-7
  9. allow case insensitive matching of file links, e.g. File:image.jpg
  10. breaks quote 
    <[http://www.example.com]>
  11. ((image.jpg)) shows images from image/ folder ???
  12. <[ ]> eats blank line in quote, undesired
  13. broken nested quote 
    • <[block
<[nested quote]>
quote
]>

11. Template Engine

  1. Templatest deliberately eats all line breaks see WORKAROUND
    • textarea issues <textarea>[ ' body ' ]</textarea>
      1. multi-line
      2. auto-indent
    • edit, comment, permission, caption, ...
  2. write templates.tpl for remaining actions and handlers where suitable DONE
    1. toc 
    2. tree

12. Cache

  • should also depend on user language DONE

13. Admin Panel

  1. add Check for Updates button in Admin panel: /Download/VersionCheck?
  2. Synchronizing data: update comment count for page if out of sync
  3. Querying the RIPE Database: https://apps.db.ripe.net/db-web-ui/#/query
    1. https://rest.db.ripe.net/search.json?query-string=
    2. user approval
    3. event log 
    4. Bad Behavior
  4. upload module 
    1. PAY ATTENTION TO SECURITY RISKS
 

Before adding random file/MIME types: please think about possible security issues.

For example HTML (.htm, .html), JavaScript (.js) and PHP (.php) file are types you’d better avoid as they can be “executed” on your server where you really would not want that to happen. For most of these kind of files, this should not be a problem though as these files are better off being compressed into a ZIP file anyway.

Only add file types that you REALLY need and that you are comfortable with.
  5. user management
    • deactivate / delete inactive users
      • criteria
      • actions
  6. add module to filter, moderate and manage pages, comments, (files)
    1. see modules for content like pages
  7. recovery mode: CSS and images won't load
    • index.php: (!$db->ap_mode && RECOVERY_MODE) excludes static files!
      • !$db->ap_mode
  8. purge logs (TRUNCATE)
    1. log table
    2. referrers
    3. badbehavior
  9. use collgroup for col span % width 
    • <colgroup>
  <col span="1" style="width: 10%;">
  <col span="1" style="width: 5%;">
  <col span="1" style="width: 5%;">
  <col span="1" style="width: 45%;">
  <col span="1" style="width: 15%;">
  <col span="1" style="width: 10%;">
  <col span="1" style="width: 10%;">
</colgroup>
    •  <thead class="data-head">
        <tr class="">
        </tr>
        [...]
    </thead>

<tbody id="table-section-one">
[...]
</tbody>

<tbody id="table-section-two">
[...]
</tbody>

14. Database

  1. https://www.digitalocean.com/c[...]ur-mysql-5-7-upgrade
  2. /Dev/Guidelines/SQL/SQLmodes

14.1. Check for SQL STRICT mode violations

  1. Using GROUP BY and selecting an ambiguous column
  2. Inserting the non standard zero date into a datetime column
  3. Inserting a 20 character string into a 10 character column
  4. Division by zero
  5. Inserting a negative value into an unsigned column

so far

  1. #1406 – Data too long for column 'description' at row 1
    1. set HTML maxlength="DB_FIELDSIZE" for all (VAR)CHAR form field
      • suggested (JS hint – might differ in some cases – smaller, e.g. meta description 160, meta title 60) + database field size (mandatory enforcement)
      • JS hint: You have <strong>60</strong> characters left
    2. set PHP length check before passing to INPUT / UPDATE
  2. #1055 – 'dev.g.group_name' isn't in GROUP BY

SELECT
`DIGEST_TEXT` AS `query`,
`SCHEMA_NAME` AS `db`,
`COUNT_STAR` AS `exec_count`,
`SUM_ERRORS` AS `errors`,
(ifnull((`SUM_ERRORS` / nullif(`COUNT_STAR`,0)),0) * 100) AS `error_pct`,
`SUM_WARNINGS` AS `warnings`,
(ifnull((`SUM_WARNINGS` / nullif(`COUNT_STAR`,0)),0) * 100) AS `warning_pct`,
`FIRST_SEEN` AS `first_seen`,
`LAST_SEEN` AS `last_seen`,
`DIGEST` AS `digest`
FROM
 performance_schema.events_statements_summary_by_digest
WHERE
((`SUM_ERRORS` > 0) OR (`SUM_WARNINGS` > 0))
ORDER BY
 `SUM_ERRORS` DESC,
 `SUM_WARNINGS` DESC;

15. WikiEdit

  1. use only one popup for new link, having link and link description together
  2. popup for tables
3. select rows and columns
  1. set table header
  2. select color for text and highlighting
  3. resize textarea
  4. undo / redo
  5. javascript search&replace
  6. symbols

16. Translations

-> /Dev/Release/R5.5/Translations

17. Usability issues

  1. indicate page [language|permissions]
  2. add double-click support for editing comments
  3. When should I use a select box instead of radio buttons?
    1. https://www.nngroup.com/articl[...]es-vs-radio-buttons/
  4. Not indicating an active form field
    • e.g. 
      textarea:focus {
    border: 1px solid red; }
    • You can use the ‘:focus’ selector on lots of elements, but it’s super handy when used on inputs and textareas to indicate that the field is active. Add CSS styling such as a highlighted border, or a subtle change to the background color.
  5. forms with checkboxes and options in lists
    • e.g. category handler or users in admin panel
    • assignment of form buttons
  6. usage of new page handler
    • seen in many fresh installs, users adding sub pages to HomePage/subpage
      • this is possible but is it really desired and understood, should we filter out system pages as pre-provided cluster in the /new [page] handler?
  7. make name spaces for users and groups more intuitive accessible
    1. users/nickname/userspace/..
    2. groups/usability/groupspace/..
  8. add ability to easily create groups and to add group members
    1. suggestions?
  9. GUI inconsistencies
    1. handler
    2. actions
    3. message boxes
  10. Your session has timed out. Please sign in again.

Readings

  1. https://www.nngroup.com/articles/low-contrast/
  2. https://backchannel.com/how-th[...]eadable-a781ddc711b6

18. Libs

  1. php-diff successor
    1. https://github.com/jfcherng/php-diff
    2. https://github.com/JBlond/php-diff
    3. https://github.com/sebastianbergmann/diff

19. Extensions

20. Refactoring

  1. https://www.owasp.org/index.php/Logging_Cheat_Sheet

21. Staging Area

  1. blog action (will replace news action)
  2. snippet action
  3. forum, topics action
  4. Admin Panel
    1. refactor -> antipatterns
    2. bad behaviour module
    3. list and check each module, assign status
  5. moderate handler -> quite a mess
  6. poll actions -> quite a mess

22. Ideas

  1. spam / badword handling -> bad_words($text) function
    1. https://en.wikipedia.org/wiki/Wordfilter
    2. https://stackoverflow.com/ques[...]ood-profanity-filter
    3. What you need is a good way for users to flag inappropriate content and a mechanism to deal with it swiftly. One way is to automatically hide/remove content if it's been flagged more than X times.
  2. HTML5 media action: {{media type="[audio|video]" source=http://.... [width=000] [height=000] [...some other options...]}}
  3. rel="edit" -> https://tools.ietf.org/html/rfc686
  4. enforce ACL-Policy, e.g. set read to $, user can't overwrite the setting
  5. test PHPThumb alternatives
    1. https://github.com/mosbth/cimage
  6. Composer
  7. https://github.com/openpgpjs/openpgpjs
  8. https://highlightjs.org/
  9. https://www.w3.org/TR/css3-page/
    1. https://www.smashingmagazine.c[...]-for-print-with-css/

23. Themes


  1. Clean up themes section
    • update repo links and info section
    • foster or (re)move theme
  2. default theme:
    1. switch from pixels to (root) ems 
    2. increase global font-size from 13px to 16px
    3. add flexbox support
  3. new mobile ready theme / layout? -> on hold (we got a new template engine!)
  4. https://github.com/KDE/breeze-icons
  5. https://developers.google.com/web/fundamentals/
  6. https://www.w3.org/Style/Examples/007/leaders

Flexbox vs Grid

  1. Flexbox: content dictates layout
  2. Grid: container dictates layout (to some extent)

Flexbox is great, it just isn't the best thing for overall page layouts.
Flexbox and grid play well together, and are a huge step forward from the float & table hacks they replace. The sooner we can use them both in production, the better.

CSS Varibles

  1. https://developer.mozilla.org/[...]/Using_CSS_variables
  2. https://www.w3.org/TR/css-variables/

CSS colums for index <div class="gimme-columns"><ul>

 
.gimme-columns {
	columns: 20em 3;
	column-count: 3;
	column-width: 20em;
}

.sticky {
  position: sticky;
  top: 0;
}

default theme

  • change blockquote
    • box-shadow: 0 0 6px rgba(0, 0, 0, 0.5);
  • add option to hide $this->config['site_name'] in theme header, e.g.: WackoWiki: To Do R5.5 -> To Do R5.5
  • min-height: 200px for .article, #page

23.1. SVG

Icon setting: add viewBox="0 0 16 16" AND height="16" width="16"


explain

23.2. Site Logo


Customizable Logo and Favicon

  1. Location
2. currently image/ folder
  1. chmod issue
  2. Options
    1. Favicon
      1. site_favicon
    2. Logo
      1. site_logo
      2. logo_width
      3. logo_height

23.3. Favicon

Supported Formats

  1. ico -> type="image/x-icon" ->
  2. png -> type="image/png" -> http://caniuse.com/link-icon-png
  3. gif -> type="image/gif" ->
  4. svg -> type="image/svg+xml" -> http://caniuse.com/link-icon-svg

Dimensions

  1. 16×16
  2. 32×32
  3. 48×48
  4. 64×64

  1. https://en.wikipedia.org/wiki/Favicon
  2. https://github.com/audreyr/favicon-cheat-sheet
  3. https://html.spec.whatwg.org/m[...]antics.html#rel-icon

24. Documentation

  1. add documentation for Admin Panel
    1. backup and restore module
  2. Update / foster Core Documentation for
    1. Deutsch
    2. English
    3. Français
    4. Русский
  3. describe forum and topic action
  4. blog action -> Blogging with WackoWiki
  5. where and when you should use relative or absolute addressing (include action, files, actions with page parameter)
  6. how you use the include function (pages, comments)
  7. release notes
    1. add a New & Noteworthy section / sub page to raise visibility of 'hidden' features
  8. moderate handler
  9. message functions and usage
  10. add sub page to config documentation about CSP 
  11. Video Tutorials
  12. update screen shots

25. Feedback

  1. themes
  2. migration and encoding issues
  3. frequent annoying issues
  4. unsolved questions
  5. add rfc section in dev cluster
  6. add dev activity log 
  7. Nginx configuration (rewrite rules)

26. Testing

  1. Tools
  2. https://github.com/mozilla/readability – test with firefox reading mode
  3. /Forum/Discussion/UserPasswordReset -> solution paths

26.1. Test cases

-> Test cases

26.2. Debug

  1. Content Security Policy: Die Einstellungen der Seite haben das Laden einer Ressource auf self blockiert ("script-src http://localhost 'unsafe-inline'").
    • call to eval() or related function blocked by CSP: autocomplete.js (Line 253)

Read comment (1 comment)