std::codecvt_utf8

C++

Language

Headers

Freestanding and hosted implementations

Named requirements

Language support library

Concepts library (C++20)

Diagnostics library

Utilities library

Strings library

Containers library

Algorithms library

Iterators library

Numerics library

Input/output library

Localizations library

Regular expressions library (C++11)

Atomic operations library (C++11)

Thread support library (C++11)

Filesystem library (C++17)

Technical Specifications

Localizations library

Locales and facets
locale
use_facet	has_facet
Character classification
isspace isblank (C++11) iscntrl isupper islower isalpha	isdigit ispunct isxdigit isalnum isprint isgraph
Conversions
toupper tolower	wstring_convert (C++11) wbuffer_convert (C++11)
Facet category base classes
ctype_base codecvt_base messages_base	time_base money_base
Facet categories
ctype ctype<char> codecvt collate messages time_get time_put	num_get num_put numpunct money_get money_put moneypunct
Locale-specific facets
ctype_byname codecvt_byname messages_byname collate_byname	time_get_byname time_put_byname numpunct_byname moneypunct_byname
Code conversion facets
codecvt_utf8 (C++11) codecvt_utf16 (C++11)	codecvt_utf8_utf16 (C++11) codecvt_mode (C++11)
C locale
setlocale localeconv lconv	LC_ALLLC_COLLATELC_CTYPELC_MONETARYLC_NUMERICLC_TIME

Defined in header <codecvt>
template< class Elem, unsigned long Maxcode = 0x10ffff, std::codecvt_mode Mode = (std::codecvt_mode)0 > class codecvt_utf8 : public std::codecvt<Elem, char, std::mbstate_t>;		(since C++11) (deprecated in C++17)

std::codecvt_utf8 is a std::codecvt facet which encapsulates conversion between a UTF-8 encoded byte string and UCS2 or UTS-32 character string (depending on the type of Elem). This codecvt facet can be used to read and write UTF-8 files, both text and binary.

Template Parameters

Elem	-	either char16_t, char32_t, or wchar_t
Maxcode	-	the largest value of Elem that this facet will read or write without error
Mode	-	a constant of type std::codecvt_mode

Member functions

(constructor)	constructs a new codecvt_utf8 facet (public member function)
(destructor)	destroys a codecvt_utf8 facet (public member function)

std::codecvt_utf8::codecvt_utf8

explicit codecvt_utf8( std::size_t refs = 0 );

Constructs a new std::codecvt_utf8 facet, passes the initial reference counter refs to the base class.

Parameters

refs

-

the number of references that link to the facet

std::codecvt_utf8::~codecvt_utf8

~codecvt_utf8();

Destroys the facet. Unlike the locale-managed facets, this facet's destructor is public.

Inherited from std::codecvt

Member types

Member type	Definition
intern_type	internT
extern_type	externT
state_type	stateT

Member objects

Member name	Type
id (static)	std::locale::id

Member functions

out	invokes do_out (public member function of std::codecvt)
in	invokes do_in (public member function of std::codecvt)
unshift	invokes do_unshift (public member function of std::codecvt)
encoding	invokes do_encoding (public member function of std::codecvt)
always_noconv	invokes do_always_noconv (public member function of std::codecvt)
length	invokes do_length (public member function of std::codecvt)
max_length	invokes do_max_length (public member function of std::codecvt)

Protected member functions

do_out [virtual]	converts a string from internT to externT, such as when writing to file (virtual protected member function of std::codecvt)
do_in [virtual]	converts a string from externT to internT, such as when reading from file (virtual protected member function of std::codecvt)
do_unshift [virtual]	generates the termination character sequence of externT characters for incomplete conversion (virtual protected member function of std::codecvt)
do_encoding [virtual]	returns the number of externT characters necessary to produce one internT character, if constant (virtual protected member function of std::codecvt)
do_always_noconv [virtual]	tests if the facet encodes an identity conversion for all valid argument values (virtual protected member function of std::codecvt)
do_length [virtual]	calculates the length of the externT string that would be consumed by conversion into given internT buffer (virtual protected member function of std::codecvt)
do_max_length [virtual]	returns the maximum number of externT characters that could be converted into a single internT character (virtual protected member function of std::codecvt)

Inherited from std::codecvt_base

Member type	Definition
enum result { ok, partial, error, noconv };	Unscoped enumeration type

Enumeration constant	Definition
ok	conversion was completed with no error
partial	not all source characters were converted
error	encountered an invalid character
noconv	no conversion required, input and output types are the same

Notes

Although the standard requires that this facet works with UCS2 when the size of Elem is 16 bits, some implementations use UTF-16 instead. The term "UCS2" was deprecated and removed from the Unicode standard.

Example

The following example demonstrates the difference between UCS2/UTF-8 and UTF-16/UTF-8 conversions: the third character in the string is not a valid UCS2 character.

Run this code

#include <iostream>
#include <string>
#include <locale>
#include <codecvt>

int main()
{
    // UTF-8 data. The character U+1d10b, musical sign segno, does not fit in UCS2
    std::string utf8 = u8"z\u6c34\U0001d10b";

    // the UTF-8 / UTF-16 standard conversion facet
    std::wstring_convert<std::codecvt_utf8_utf16<char16_t>, char16_t> utf16conv;
    std::u16string utf16 = utf16conv.from_bytes(utf8);
    std::cout << "UTF16 conversion produced " << utf16.size() << " code units:\n";
    for (char16_t c : utf16)
        std::cout << std::hex << std::showbase << c << '\n';

    // the UTF-8 / UCS2 standard conversion facet
    std::wstring_convert<std::codecvt_utf8<char16_t>, char16_t> ucs2conv;
    try {
        std::u16string ucs2 = ucs2conv.from_bytes(utf8);
    } catch(const std::range_error& e) {
        std::u16string ucs2 = ucs2conv.from_bytes(utf8.substr(0, ucs2conv.converted()));
        std::cout << "UCS2 failed after producing " << std::dec << ucs2.size()<<" characters:\n";
        for (char16_t c : ucs2)
            std::cout << std::hex << std::showbase << c << '\n';
    }
}

Output:

UTF16 conversion produced 4 code units:
0x7a
0x6c34
0xd834
0xdd0b
UCS2 failed after producing 2 characters:
0x7a
0x6c34

See also

Character conversions	locale-defined multibyte (UTF-8, GB18030)	UTF-8	UTF-16
UTF-16	mbrtoc16 / c16rtomb(with C11's DR488)	codecvt<char16_t, char, mbstate_t> codecvt_utf8_utf16<char16_t> codecvt_utf8_utf16<char32_t> codecvt_utf8_utf16<wchar_t>	N/A
UCS2	c16rtomb(without C11's DR488)	codecvt_utf8<char16_t> codecvt_utf8<wchar_t>(Windows)	codecvt_utf16<char16_t> codecvt_utf16<wchar_t>(Windows)
UTF-32	mbrtoc32 / c32rtomb	codecvt<char32_t, char, mbstate_t> codecvt_utf8<char32_t> codecvt_utf8<wchar_t>(non-Windows)	codecvt_utf16<char32_t> codecvt_utf16<wchar_t>(non-Windows)
system wide: UTF-32(non-Windows) UCS2(Windows)	mbsrtowcs / wcsrtombs use_facet<codecvt <wchar_t, char, mbstate_t>>(locale)	No	No

codecvt	converts between character encodings, including UTF-8, UTF-16, UTF-32 (class template)
codecvt_mode (C++11)(deprecated in C++17)	tags to alter behavior of the standard codecvt facets (enum)
codecvt_utf16 (C++11)(deprecated in C++17)	converts between UTF-16 and UCS2/UCS4 (class template)
codecvt_utf8_utf16 (C++11)(deprecated in C++17)	converts between UTF-8 and UTF-16 (class template)