kn ITN: add cardinal semiotic class#455
Conversation
Signed-off-by: arya24-mr <aryam@nvidia.com>
folivoramanh
left a comment
folivoramanh
left a comment
There was a problem hiding this comment.
- add init into data/numbers
- remove nemo_text_processing 2
- add sparrowhawk file and run sh test (we need to make sure both pytest and sparrowhawk pass)
- add kn to jenkins file
There was a problem hiding this comment.
is this 1 - 99? why we need to list manually like this
| ೯೬ ತೊಂಬತ್ತಾರು | ||
| ೯೭ ತೊಂಬತ್ತೇಳು | ||
| ೯೮ ತೊಂಬತ್ತೆಂಟು | ||
| ೯೯ ತೊಂಬತ್ತೊಂಬತ್ತು No newline at end of file |
There was a problem hiding this comment.
same question, why we need manual listing 90 rows?
| @@ -0,0 +1,13 @@ | |||
| # Copyright (c) 2024, NVIDIA CORPORATION. All rights reserved. | |||
There was a problem hiding this comment.
please update to 2026
| @@ -0,0 +1,13 @@ | |||
| # Copyright (c) 2024, NVIDIA CORPORATION. All rights reserved. | |||
| @@ -0,0 +1,181 @@ | |||
| # Copyright (c) 2024, NVIDIA CORPORATION. All rights reserved. | |||
| # Copyright 2024 and onwards Google, Inc. | |||
There was a problem hiding this comment.
2026 and delete line 2, you can check Spanish or Portuguese to copy
| ಇಪ್ಪತ್ತಮೂರು ಲಕ್ಷ ನಲವತ್ತೈದು ಸಾವಿರ ಐದು ನೂರು ಅರವತ್ತೇಳು~೨೩೪೫೫೬೭ | ||
| ಒಂದು ಕೋಟಿ ಇಪ್ಪತ್ತಮೂರು ಲಕ್ಷ ನಲವತ್ತೈದು ಸಾವಿರ ಐದು ನೂರು ಅರವತ್ತೇಳು~೧೨೩೪೫೫೬೭ | ||
| ಒಂದು ಕೋಟಿ ಇಪ್ಪತ್ತೊಂದು ಲಕ್ಷ ಇಪ್ಪತ್ತೊಂದು ಸಾವಿರ ಎರಡು ನೂರು ಹನ್ನೆರಡು~೧೨೧೨೧೨೧೨ | ||
| ಏಳೂವರೆ ನೂರು~೭೫೦ |
There was a problem hiding this comment.
please test both en number and kn number, and make the test case varied
eg: 123456789
There was a problem hiding this comment.
do not add blind test set here, we only publish unit test
| # graph_exception = pynini.union(*labels_exception).optimize() | ||
|
|
||
| self.graph_no_exception = graph | ||
| self.graph = graph # (pynini.project(graph, "input") - graph_exception.arcsort()) @ graph |
There was a problem hiding this comment.
2 graph with same function?
| ) | ||
|
|
||
| # Specialized component rules for distinct fraction terms | ||
| graph_fraction_idioms = pynini.union( |
There was a problem hiding this comment.
avoid hardcoding on .py file, let's add it to tsv and call when use it
|
|
||
|
|
||
| class CardinalFst(GraphFst): | ||
|
|
| from nemo_text_processing.text_normalization.pt.verbalizers.verbalize_final import VerbalizeFinalFst | ||
| elif lang == 'ko': | ||
| from nemo_text_processing.text_normalization.ko.taggers.tokenize_and_classify import ClassifyFst | ||
| from nemo_text_processing.text_normalization.ko.verbalizers.verbalize_final import VerbalizeFinalFst |
There was a problem hiding this comment.
please double check with main branch
|
I will review and update all the changes. |
3 participants
What does this PR do ?
This is a PR for Kannada language ITN for cardinal semiontic class
Before your PR is "Ready for review"
Pre checks:
git commit -sto sign.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files?Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files?Copyright 2015 and onwards Google, Inc.. See an example here.try import: ... except: ...) if not already done.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.