MeridianMERIDIAN

Emoji

representation.text.emoji

Single emoji character or emoji sequence, including keycap sequences (a digit, # or * with an optional variation selector and U+20E3).

Try a value

Emoji's rule accepts it.

Reads it as 😀

Use it

Shell
finetype infer -i '😀' --mode column
# → representation.text.emoji
DuckDB
SELECT ft_infer('😀');
-- → 'representation.text.emoji'
Cast template: {col} is your column
CAST({col} AS VARCHAR)
Casts to
VARCHAR
Standard

Pattern

Regular expression
^(?:[0-9#*]️?⃣|[\p{So}\p{Sk}\p{Mn}\p{Cf}])+$
ValueLengthShape
😀1not checked
🎉1not checked
❤️2not checked
🚀1not checked
1️⃣3not checked
#️⃣3not checked

pattern uses syntax this checker does not support

Background

Origin

The word emoji comes from Japanese e ('picture') + moji ('character').[1] The resemblance of the word emoji to the English words emotion and emoticon is purely coincidental.[1] Emoji became available in 1999 on Japanese mobile phones.[2] In October 2016, the Museum of Modern Art acquired the original collection of emoji distributed by NTT DoCoMo in 1999.[1]

An early proposal in 2000 to encode DoCoMo emoji in the Unicode Standard led to no action being taken.[2] Each Japanese mobile phone carrier developed a different, but partially overlapping, set of emoji.[2] Each Japanese mobile phone vendor used its own text encoding extensions, which were incompatible with one another.[2] On Japanese phones, characters from other platforms that could not be displayed were represented with 〓 (U+3013 GETA MARK).[2] A set of 722 characters was defined as the union of emoji characters used by Japanese mobile phone carriers.[2]

Unicode Technical Standard #51, Unicode Emoji, defines the structure of Unicode emoji characters and sequences.[2] Version 18.0 of UTS #51 is dated 2026-08-28.[2] UTS #51 defines an emoji as a colorful pictograph that can be used inline in text.[2] Starting with Version 11.0 of UTS #51, the repertoire of emoji characters is synchronised with the Unicode Standard and has the same version numbering.[2]

Use

Under UTS #51, an emoji is represented internally as an image, an encoded character, or a sequence of encoded characters.[2] UTS #51 defines U+FE0F VARIATION SELECTOR-16 as the character used to request an emoji presentation for an emoji character.[2] UTS #51 defines U+FE0E VARIATION SELECTOR-15 as the character used to request a text presentation for an emoji character.[2] UTS #51 defines an emoji flag sequence as a sequence of two Regional Indicator characters.[2] UTS #51 states that a singleton Regional Indicator character is not a well-formed emoji flag sequence.[2] UTS #51 defines an emoji ZWJ sequence as an emoji sequence with at least one joiner character, U+200D.[2] Five symbol modifier characters providing a range of skin tones for human emoji were released in Unicode Version 8.0.[2] The skin-tone modifiers are based on the six tones of the Fitzpatrick scale, a recognised standard for dermatology.[2] UTS #51 notes that all emoji sequences are single grapheme clusters, with never a grapheme cluster boundary inside one.[2]

In JavaScript, a string's length property counts UTF-16 code units.[3] In DuckDB, length(string) returns the number of characters in a string.[4] In DuckDB, length_grapheme(string) returns the number of grapheme clusters in a string.[4] DuckDB's documentation gives length_grapheme('🤦🏼‍♂️🤦🏽‍♀️') as 2.[4] Decide which count you mean, bytes, code points or grapheme clusters, before you set a column width or a length check on emoji data. (Meridian's view)

MySQL's utf8mb3 character set supports BMP characters only, with no support for supplementary characters.[5] MySQL's utf8mb3 character set is deprecated.[5] If emoji go missing or arrive mangled after a load into a database, the column's character set is the first place to look. (Meridian's view)

UTS #51 gives a simplified expression that can be used to quickly scan for possible emoji.[2] UTS #51 states that the only valid emoji flag sequences are those listed in the emoji-sequences.txt data file.[2] Unicode's emoji-test.txt file gives each emoji a status of component, fully-qualified, minimally-qualified or unqualified.[6] To validate strictly, check values against the emoji-test.txt of the Unicode version your source system supports, rather than against character categories alone. (Meridian's view) When grouping or joining on an emoji column, decide deliberately whether a value with U+FE0F and the same value without it count as one emoji. (Meridian's view)

Sources

  1. [1]Emoji - Wikipediasecondary source
  2. [2]UTS #51: Unicode Emojistandard
  3. [3]String: length - JavaScript | MDNdocumentation
  4. [4]Text Functions – DuckDBdocumentation
  5. [5]12.9.2 The utf8mb3 Character Set (3-Byte UTF-8 Unicode Encoding)documentation
  6. [6]emoji-test.txtstandard

How it's detected

Single emoji or emoji sequence. Pattern matches Unicode symbol categories. Emoji width variable. Keycap sequences (UTS #51: a digit, # or * then an optional U+FE0F and U+20E3) are accepted in both the fully-qualified form and the unqualified form without U+FE0F. A bare digit, # or * is not an emoji.

Schema

https://meridian.online/schemas/representation.text.emoji · Frictionless type string

finetype taxonomy representation.text.emoji -o json-schema

Type Registry