View source for UTF-8 Action Plan

{{toc numerate=1}}

===PHP===
#|
*| Version | Release | Supported until | Notes |*
|| 7.4 | December 2019 | December 2022 | Typed Properties 2.0, Preloading, Null Coalescing Assignment Operator, Improve openssl_random_pseudo_bytes, Weak References, FFI – Foreign Function Interface ||
|| 8.0 | Q1 2021 | Q1 2024 | Just In Time compilation (JIT), arrays starting with a negative index, consistent type errors for internal functions, fatal error for incompatible method signatures ||
|#

((https://www.php.net/supported-versions.php Supported Versions))
((/Dev/Guidelines/UnicodeCheatSheet Unicode Cheat Sheet))

===Questions===
====Where do we need Unicode support?====
  1. all text fields in the database
  2. all text files

====Where do we need Normalization?====
  1. tag
  2. file_name
  3. user_name

((Normalization))

====Where do we need Transliteration support?====

  1. file names for attachments
    * optional ((Transliteration transliteration)) via ##upload_translit## setting
    * All the control characters and Unicode ones should be removed from the filenames and their extensions without any exception. Also, the special characters such as "##;##", "##:##", "##>##", "##<##", "##/##" ,"##\##", additional "##.##", "##*##", "##%##", "##$##", and so on should be discarded as well. If it is applicable and there is no need to have Unicode characters, it is highly recommended to only accept Alpha-Numeric characters and only 1 dot as an input for the file name and the extension; in which the file name and also the extension should not be empty at all (regular expression: ##[a-zA-Z0-9]{1,200}\.[a-zA-Z0-9]{1,10}##).
  2. user names
  2. supertag (droped)
    * supertag is no longer needed and suited as unique identifier

((Transliteration))

====Why you may want to limit allowed characters?====
%%
if (preg_match('/[^\p{Common}\p{Latin}]/u', $_POST['text']))
{
	// post your data to the database.
}
%%

===Core===
  * convert all PHP files to UTF-8 - DONE
  * set utf-8 as default encoding  - DONE
  * remove do_unicode_entities() calls - DONE
  * remove all supertag usage - DONE
  * Templatest filters -> Unicode

===Database===
  * drop support for MyISAM. - DONE
  * drop supertag fields - DONE
  * ((/Dev/Release/R6.0/Unicode/utf8mb4 utf8mb4))
  * ((/Dev/Release/R6.0/Upgrade/DatabaseConversion Database UTF-8 conversion script)) - DONE


===Libraries===
  * php-diff - add unicode support - DONE
  * diff - DONE

===Migration===
🍏🍎🙂🐉

===Testing===
  * notifications with diff - DONE