Skip to content

Commit

Permalink
Fix merge conflicts
Browse files Browse the repository at this point in the history
  • Loading branch information
kipcole9 committed Apr 20, 2024
2 parents f6112a8 + 449bf15 commit d200297
Show file tree
Hide file tree
Showing 1,015 changed files with 1,790 additions and 111,460 deletions.
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,29 @@

**Note that `ex_cldr` version 2.33.0 and later are supported on Elixir 1.11 and later only.**

## Cldr v2.38.0

This is the changelog for Cldr v2.38.0 released on April 21st, 2023. For older changelogs please consult the release tag on [GitHub](https://github.com/elixir-cldr/cldr/tags)

### Enhancements

* Update to [CLDR 45.0](https://cldr.unicode.org/index/downloads/cldr-45) data.

* Adds `Cldr.validate_locale!/2`. Thanks to @jarrodmoldrich for the suggestion.

* Add decimal separator and grouping separator to the currency data for each locale. In some rare cases, like the currency [CVE](https://en.wikipedia.org/wiki/Cape_Verdean_escudo#:~:text=The%20escudo%20(sign%3A%20%3B%20ISO,subdivided%20into%20one%20hundred%20centavos.) in the locale [pt-CV](https://www.localeplanet.com/icu/pt-CV/index.html) the currency symbol is placed where decimal separator is normally placed. The same can apply for the grouping separator although it appears not locale uses this field.

* Adjust the `Inspect` protocol implementation for `t:Cldr.LanguageTag.t/0` types. When the language tag is resolved to a CLDR locale then the output is executable code. For example:

```elixir
iex> MyApp.Cldr.Locale.new!("en-US")
MyApp.Cldr.Locale.new!("en-US")
````

* Fix dialyzer warnings. Thanks to @Munksgaard for the PR. Closes #220. Also fixes `:underspecs` warning and the `:underspecs` dialyzer flag is now configured.

* Add configuration for the new [ex_cldr_person_names](https://hex.pm/packages/ex_cldr_person_names) backend module generator.

## Cldr v2.37.5

This is the changelog for Cldr v2.37.5 released on November 2nd, 2023. For older changelogs please consult the release tag on [GitHub](https://github.com/elixir-cldr/cldr/tags)
Expand Down Expand Up @@ -32,6 +55,10 @@ This is the changelog for Cldr v2.37.3 released on Octoebr 12th, 2023. For olde

* Adds `<backend>.put_gettext_locale/1`.

* Adds `Cldr.validate_locale!/2`. Thanks to @jarrodmoldrich for the suggestion.

* Add decimal separator and grouping separator to the currency data for each locale. In some rare cases, like the currency [CVE](https://en.wikipedia.org/wiki/Cape_Verdean_escudo#:~:text=The%20escudo%20(sign%3A%20%3B%20ISO,subdivided%20into%20one%20hundred%20centavos.) in the locale [pt-CV](https://www.localeplanet.com/icu/pt-CV/index.html) the currency symbol is placed where decimal separator is normally placed. The same can apply for the grouping separator although it appears not locale uses this field.

## Cldr v2.37.2

This is the changelog for Cldr v2.37.2 released on July 4th, 2023. For older changelogs please consult the release tag on [GitHub](https://github.com/elixir-cldr/cldr/tags)
Expand Down
4 changes: 2 additions & 2 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,9 +142,9 @@ git pull
mix cldr.consolidate
```

3. Then regenerate the `language_tags.ebin` file by executing the following:
3. Then regenerate the `language_tags.ebin` file by executing the following making sure to set the `DEV` shell variable which disables locale stale tests:
```bash
mix cldr.generate_language_tags
DEV=true mix cldr.generate_language_tags
```


4 changes: 0 additions & 4 deletions ldml2json
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ mkdir -p $CLDR_PRODUCTION/bcp47
cp -R $CLDR_REPO/common/bcp47/ $CLDR_PRODUCTION/bcp47/

# ex_cldr additional data required for data generation
cp $EX_CLDR/data/measurement_systems.xml $CLDR_PRODUCTION/measurement_systems.xml
cp $EX_CLDR/data/iso_currencies.xml $CLDR_PRODUCTION/iso_currencies.xml

# Data that maps from Unicode script names to Script code (subtag)
Expand All @@ -100,7 +99,4 @@ cp $CLDR_REPO/common/testData/units/unitPreferencesTest.txt \
cp $CLDR_REPO/common/testData/units/unitsTest.txt \
$EX_CLDR/test/data/conversion_test_data.txt

cp -R $CLDR_REPO/common/testData/personNameTest \
$EX_CLDR/test/data/person_name_test_data

cd $OLDPWD
144 changes: 105 additions & 39 deletions lib/cldr.ex
Original file line number Diff line number Diff line change
Expand Up @@ -304,10 +304,13 @@ defmodule Cldr do
iex> import Cldr.LanguageTag.Sigil
iex> Cldr.put_gettext_locale(~l"de")
{:error,
{Cldr.UnknownLocaleError,
"Locale #Cldr.LanguageTag<de [validated]> does not map to a known gettext locale name"}}
{
:error,
{
Cldr.UnknownLocaleError,
"Locale TestBackend.Cldr.Locale.new!(\\"de-DE\\") does not map to a known gettext locale name"
}
}
"""
@spec put_gettext_locale(LanguageTag.t()) ::
{:ok, binary() | nil} | {:error, {module(), String.t()}}
Expand Down Expand Up @@ -395,7 +398,7 @@ defmodule Cldr do
"""
@doc since: "2.32.0"

@spec with_locale(Locale.locale_reference(), fun) :: any
@spec with_locale(Locale.locale_reference(), (-> any())) :: any
def with_locale(%Cldr.LanguageTag{} = locale, fun) when is_function(fun) do
current_locale = get_locale(locale.backend)

Expand Down Expand Up @@ -433,7 +436,7 @@ defmodule Cldr do
"""
@doc since: "2.27.0"

@spec with_locale(Locale.locale_reference(), backend(), fun) :: any
@spec with_locale(Locale.locale_reference(), backend(), (-> any())) :: any()
def with_locale(locale, backend \\ default_backend!(), fun) when is_locale_name(locale) do
with {:ok, locale} = validate_locale(locale, backend) do
with_locale(locale, fun)
Expand Down Expand Up @@ -671,7 +674,7 @@ defmodule Cldr do
@doc """
Return a localised string suitable for
presentation purposes for structs that
implement the `Cldr.LanguageTag.DisplayName`
implement the `Cldr.LanguageTag.DisplayName`
protocol.
The `Cldr.LanguageTag.DisplayName` protocol is
Expand Down Expand Up @@ -807,7 +810,7 @@ defmodule Cldr do
{:error, {Cldr.InvalidLanguageError, "The language \\"zzz\\" is invalid"}}
"""
@spec validate_locale(Locale.locale_name() | LanguageTag.t() | String.t(), backend()) ::
@spec validate_locale(Locale.locale_reference(), backend()) ::
{:ok, LanguageTag.t()} | {:error, {module(), String.t()}}

def validate_locale(locale, backend \\ nil)
Expand All @@ -824,6 +827,77 @@ defmodule Cldr do
backend.validate_locale(locale)
end

@doc """
Normalise and validate a locale name or raises
an exception.
## Arguments
* `locale` is any valid locale name returned by `Cldr.known_locale_names/1`
or a `Cldr.LanguageTag` struct returned by `Cldr.Locale.new!/2`
* `backend` is any module that includes `use Cldr` and therefore
is a `Cldr` backend module. The default is `Cldr.default_backend!/0`.
Note that `Cldr.default_backend!/0` will raise an exception if
no `:default_backend` is configured under the `:ex_cldr` key in
`config.exs`.
## Returns
* `language_tag` or
* raises an exception
## Examples
iex> Cldr.validate_locale!(:en, TestBackend.Cldr)
%Cldr.LanguageTag{
backend: TestBackend.Cldr,
canonical_locale_name: "en",
cldr_locale_name: :en,
extensions: %{},
gettext_locale_name: "en",
language: "en",
locale: %{},
private_use: [],
rbnf_locale_name: :en,
requested_locale_name: "en",
script: :Latn,
territory: :US,
transform: %{},
language_variants: []
}
iex> Cldr.validate_locale!(:af, TestBackend.Cldr)
%Cldr.LanguageTag{
backend: TestBackend.Cldr,
canonical_locale_name: "af",
cldr_locale_name: :af,
extensions: %{},
gettext_locale_name: nil,
language: "af",
locale: %{},
private_use: [],
rbnf_locale_name: :af,
requested_locale_name: "af",
script: :Latn,
territory: :ZA,
transform: %{},
language_variants: []
}
iex> Cldr.validate_locale!("zzz", TestBackend.Cldr)
** (Cldr.InvalidLanguageError) The language "zzz" is invalid
"""
@spec validate_locale!(Locale.locale_reference(), backend()) :: LanguageTag.t() | no_return()
def validate_locale!(locale, backend \\ nil) do
case validate_locale(locale, backend) do
{:ok, locale} -> locale
{:error, {module, reason}} -> raise module, reason
end
end

@doc """
Returns a list of all the locale names defined in
the CLDR repository.
Expand Down Expand Up @@ -856,7 +930,7 @@ defmodule Cldr do
The list is the combination of configured locales,
`Gettext` locales and the default locale.
See also `known_locales/1` and `all_locales/0`
See also `known_locales/1` and `all_locales/0`.
"""
@spec requested_locale_names(backend()) :: [Locale.locale_name(), ...] | []
Expand Down Expand Up @@ -1448,8 +1522,14 @@ defmodule Cldr do
@known_territories
end

@doc """
Returns a map of territories and the territories
within which they are contained.
"""
@territory_containment Cldr.Config.territory_containment()
@spec territory_containment() :: map()
@spec territory_containment() ::
unquote(Cldr.Type.territory_containment(@territory_containment))
def territory_containment do
@territory_containment
end
Expand Down Expand Up @@ -1520,7 +1600,7 @@ defmodule Cldr do
end)
|> Map.new()

@spec known_territory_subdivision_containment :: map()
@spec known_territory_subdivision_containment :: unquote(Cldr.Type.subdivision_containment())
def known_territory_subdivision_containment do
@territory_subdivision_containment
end
Expand Down Expand Up @@ -1877,13 +1957,13 @@ defmodule Cldr do
:SYP, :SZL, :THB, :TJR, :TJS, :TMM, :TMT, :TND, :TOP, :TPE, :TRL, :TRY, :TTD,
:TWD, :TZS, :UAH, :UAK, :UGS, :UGX, :USD, :USN, :USS, :UYI, :UYP, :UYU, :UYW,
:UZS, :VEB, :VED, :VEF, :VES, :VND, :VNN, :VUV, :WST, :XAF, :XAG, :XAU, :XBA,
:XBB, :XBC, :XBD, :XCD, :XDR, :XEU, :XFO, :XFU, :XOF, :XPD, :XPF, :XPT, :XRE,
:XSU, :XTS, :XUA, :XXX, :YDD, :YER, :YUD, :YUM, :YUN, :YUR, :ZAL, :ZAR, :ZMK,
:ZMW, :ZRN, :ZRZ, :ZWD, :ZWL, :ZWR]
:XBB, :XBC, :XBD, :XCD, :XCG, :XDR, :XEU, :XFO, :XFU, :XOF, :XPD, :XPF, :XPT,
:XRE, :XSU, :XTS, :XUA, :XXX, :YDD, :YER, :YUD, :YUM, :YUN, :YUR, :ZAL, :ZAR,
:ZMK, :ZMW, :ZRN, :ZRZ, :ZWD, :ZWL, :ZWR]
"""
@known_currencies Cldr.Config.known_currencies()
@spec known_currencies :: [atom(), ...] | []
@spec known_currencies :: [atom(), ...]
def known_currencies do
@known_currencies
end
Expand Down Expand Up @@ -2010,7 +2090,7 @@ defmodule Cldr do
"""
@known_number_systems Cldr.Config.known_number_systems()
@spec known_number_systems :: [atom(), ...] | []
@spec known_number_systems :: [atom(), ...]
def known_number_systems do
@known_number_systems
end
Expand Down Expand Up @@ -2343,36 +2423,22 @@ defmodule Cldr do
"The measurement system \\"something\\" is invalid"}}
"""
@measurement_systems Cldr.Config.measurement_systems()

def validate_measurement_system(system) when is_binary(system) do
system
|> String.downcase()
|> do_validate_measurement_system
end

def validate_measurement_system(system) when is_atom(system) do
do_validate_measurement_system(system)
end

@measurement_systems Cldr.Config.measurement_systems()
|> Enum.flat_map(fn
{k, %{alias: nil}} -> [{k, k}]
{k, %{alias: a}} -> [{k, k}, {a, k}]
end)
|> Map.new()

for {system, canonical_system} <- @measurement_systems do
defp do_validate_measurement_system(unquote(system)),
do: {:ok, unquote(canonical_system)}

defp do_validate_measurement_system(unquote(Kernel.to_string(system))),
do: {:ok, unquote(canonical_system)}
|> String.to_existing_atom()
|> validate_measurement_system()
rescue ArgumentError ->
{:error, unknown_measurement_system_error(system)}
end

defp do_validate_measurement_system(measurement_system) do
{:error, unknown_measurement_system_error(measurement_system)}
def validate_measurement_system(system) when system in @measurement_systems do
{:ok, system}
end

def unknown_measurement_system_error(measurement_system) do
defp unknown_measurement_system_error(measurement_system) do
{
Cldr.UnknownMeasurementSystemError,
"The measurement system #{inspect(measurement_system)} is invalid"
Expand Down
19 changes: 13 additions & 6 deletions lib/cldr/backend/cldr_backend.ex
Original file line number Diff line number Diff line change
Expand Up @@ -378,9 +378,13 @@ defmodule Cldr.Backend do
iex> import Cldr.LanguageTag.Sigil
iex> #{inspect(__MODULE__)}.put_gettext_locale(~l"de")
{:error,
{Cldr.UnknownLocaleError,
"Locale #Cldr.LanguageTag<de [validated]> does not map to a known gettext locale name"}}
{
:error,
{
Cldr.UnknownLocaleError,
"Locale TestBackend.Cldr.Locale.new!(\\"de-DE\\") does not map to a known gettext locale name"
}
}
"""
@doc since: "2.38.0"
Expand Down Expand Up @@ -414,7 +418,7 @@ defmodule Cldr.Backend do
"""
@doc since: "2.32.0"

@spec with_locale(Locale.locale_reference(), fun) :: any
@spec with_locale(Locale.locale_reference(), (-> any())) :: any
def with_locale(%Cldr.LanguageTag{} = locale, fun) when is_function(fun) do
Cldr.with_locale(locale, fun)
end
Expand Down Expand Up @@ -719,8 +723,10 @@ defmodule Cldr.Backend do
# in CLDR
#
# 1. Adjust the escaping of "\" to suit
# 2. Remove compound patterns like `{Rs}` which
# are not supported in Erlang's re
# 2. Remove spaces in the regex since `re` considers them
# part of the pattern.
# 3. Remove compound patterns like `{Rs}` which
# are not supported in Erlang's `re`.

@remove_compounds Regex.compile!("{.*}", [:ungreedy])

Expand All @@ -734,6 +740,7 @@ defmodule Cldr.Backend do
regex =
v
|> String.replace("\x5c\x5c", "\x5c")
|> String.replace(" ", "")
|> String.replace(@remove_compounds, "")

{k, Regex.compile!(regex, "u")}
Expand Down
3 changes: 2 additions & 1 deletion lib/cldr/backend/locale.ex
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ defmodule Cldr.Locale.Backend do
def define_locale_backend(config) do
known_locale_names = Cldr.Locale.Loader.known_locale_names(config)

quote location: :keep, bind_quoted: [config: Macro.escape(config), known_locale_names: known_locale_names] do
quote location: :keep,
bind_quoted: [config: Macro.escape(config), known_locale_names: known_locale_names] do
defmodule Locale do
@moduledoc false
if Cldr.Config.include_module_docs?(config.generate_docs) do
Expand Down
Loading

0 comments on commit d200297

Please sign in to comment.