Paver i18n_robot_pull Command doesn't work

bi_goutas 2019-01-20 12:45:28 UTC #1

Keywords: OpenEdX - AWS - Technical issue - Other
Description:
Hello ,

I have changed the language on openedx to French. When I select English from the language selector it gives me 500 errors while the French and Arabic versions work perfectly.

the command /opt/bitnami/apps/edx/bin/paver.edxapp i18n_robot_pull gives me this error
`INFO:i18n.generate:Copying mapped locale /opt/bitnami/apps/edx/edx-platform/conf/locale/zh_CN/LC_MESSAGES to /opt/bitnami/apps/edx/edx-platform/conf/locale/zh_HANS/LC_MESSAGES
python manage.py lms --settings=devstack compilejsi18n
Traceback (most recent call last):
File "manage.py", line 118, in
startup.run()
File "/opt/bitnami/apps/edx/edx-platform/lms/startup.py", line 19, in run
django.setup()
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/apps/config.py", line 94, in create
module = import_module(entry)
File "/opt/bitnami/python/lib/python2.7/importlib/__init__.py", line 37, in import_module
import(name)
ImportError: No module named debug_toolbar

Captured Task Output:
---> pavelib.i18n.i18n_robot_pull
---> pavelib.i18n.i18n_clean
git clean -fdX conf/locale
---> pavelib.i18n.i18n_transifex_pull
---> pavelib.i18n.i18n_validate_transifex_config
i18n_tool transifex pull
---> pavelib.i18n.i18n_extract
---> pavelib.prereqs.install_prereqs
---> pavelib.i18n.i18n_validate_gettext
i18n_tool extract
---> pavelib.i18n.i18n_dummy
i18n_tool dummy
i18n_tool generate
python manage.py lms --settings=devstack compilejsi18n

Build failed running pavelib.i18n.i18n_robot_pull: Subprocess return code: 1`

I tried to compile it with /opt/bitnami/apps/edx/bin/python.edxapp manage.py lms compilejsi18n --settings=aws and it doesn't work also
2019-01-20 05:44:44,017 WARNING 8001 [openedx.core.djangoapps.monitoring_utils.middleware] middleware.py:24 - Unable to load NewRelic agent module 2019-01-20 05:44:44,772 INFO 8001 [dd.dogapi] dog_stats_api.py:66 - Initializing dog api to use statsd: localhost, 8125 processing language en Traceback (most recent call last): File "manage.py", line 121, in execute_from_command_line([sys.argv[0]] + django_args) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line utility.execute() File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, **cmd_options) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute output = self.handle(*args, **options) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/statici18n/management/commands/compilejsi18n.py", line 108, in handle self.createoutput(outputdir, outputformat, locale, domain, packages) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/statici18n/management/commands/compilejsi18n.py", line 73, in createoutput data = self.createjavascript_catalog(locale, domain, packages) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/statici18n/management/commands/compilejsi18n.py", line 48, in createjavascript_catalog activate(locale) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/__init__.py", line 166, in activate return _trans.activate(language) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 239, in activate _active.value = translation(language) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 228, in translation _translations[language] = DjangoTranslation(language) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 131, in init self.addlocal_translations() File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 183, in addlocal_translations translation = self.newgnu_trans(localedir) File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 156, in newgnu_trans fallback=use_null_fallback) File "/opt/bitnami/python/lib/python2.7/gettext.py", line 554, in translation t = translations.setdefault(key, class(fp)) File "/opt/bitnami/python/lib/python2.7/gettext.py", line 255, in init self._parse(fp) File "/opt/bitnami/python/lib/python2.7/gettext.py", line 391, in _parse self.plural = c2py(plural) File "/opt/bitnami/python/lib/python2.7/gettext.py", line 177, in c2py result, nexttok = parse(tokenize(plural)) File "/opt/bitnami/python/lib/python2.7/gettext.py", line 114, in _parse nexttok = next(tokens) File "/opt/bitnami/python/lib/python2.7/gettext.py", line 91, in _tokenize raise ValueError('invalid token in plural form: %s' % value) ValueError: invalid token in plural form: EXPRESSION

Command paver i18n_robot_pull doesn't work

Command paver i18n_robot_pull and compilejsi18n doesn't work

Paver i18n_robot_pull got error

bi_goutas 2019-01-21 10:51:47 UTC #2

Type: Stacksmith
Description:
Hello ,

I have changed the language on openedx to French. When I select English from the language selector it gives me 500 errors while the French and Arabic versions work perfectly.

the command /opt/bitnami/apps/edx/bin/paver.edxapp i18n_robot_pull gives me this error

INFO:i18n.generate:Copying mapped locale /opt/bitnami/apps/edx/edx-platform/conf/locale/zh_CN/LC_MESSAGES to /opt/bitnami/apps/edx/edx-platform/conf/locale/zh_HANS/LC_MESSAGES
python manage.py lms --settings=devstack compilejsi18n
Traceback (most recent call last):
File "manage.py", line 118, in
startup.run()
File "/opt/bitnami/apps/edx/edx-platform/lms/startup.py", line 19, in run
django.setup()
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/apps/config.py", line 94, in create
module = import_module(entry)
File "/opt/bitnami/python/lib/python2.7/importlib/__init__.py", line 37, in import_module
import(name)
ImportError: No module named debug_toolbar

Captured Task Output:

---> pavelib.i18n.i18n_robot_pull
---> pavelib.i18n.i18n_clean
git clean -fdX conf/locale
---> pavelib.i18n.i18n_transifex_pull
---> pavelib.i18n.i18n_validate_transifex_config
i18n_tool transifex pull
---> pavelib.i18n.i18n_extract
---> pavelib.prereqs.install_prereqs
---> pavelib.i18n.i18n_validate_gettext
i18n_tool extract
---> pavelib.i18n.i18n_dummy
i18n_tool dummy
i18n_tool generate
python manage.py lms --settings=devstack compilejsi18n

Build failed running pavelib.i18n.i18n_robot_pull: Subprocess return code: 1

I tried to compile it with /opt/bitnami/apps/edx/bin/python.edxapp manage.py lms compilejsi18n --settings=aws and it doesn't work also

2019-01-20 05:44:44,017 WARNING 8001 [openedx.core.djangoapps.monitoring_utils.middleware] middleware.py:24 - Unable to load NewRelic agent module
2019-01-20 05:44:44,772 INFO 8001 [dd.dogapi] dog_stats_api.py:66 - Initializing dog api to use statsd: localhost, 8125
processing language en
Traceback (most recent call last):
File "manage.py", line 121, in
execute_from_command_line([sys.argv[0]] + django_args)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
utility.execute()
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/statici18n/management/commands/compilejsi18n.py", line 108, in handle
self.createoutput(outputdir, outputformat, locale, domain, packages)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/statici18n/management/commands/compilejsi18n.py", line 73, in createoutput
data = self.createjavascript_catalog(locale, domain, packages)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/statici18n/management/commands/compilejsi18n.py", line 48, in createjavascript_catalog
activate(locale)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/__init__.py", line 166, in activate
return _trans.activate(language)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 239, in activate
_active.value = translation(language)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 228, in translation
_translations[language] = DjangoTranslation(language)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 131, in init
self.addlocal_translations()
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 183, in addlocal_translations
translation = self.newgnu_trans(localedir)
File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 156, in newgnu_trans
fallback=use_null_fallback)
File "/opt/bitnami/python/lib/python2.7/gettext.py", line 554, in translation
t = translations.setdefault(key, class(fp))
File "/opt/bitnami/python/lib/python2.7/gettext.py", line 255, in init
self._parse(fp)
File "/opt/bitnami/python/lib/python2.7/gettext.py", line 391, in _parse
self.plural = c2py(plural)
File "/opt/bitnami/python/lib/python2.7/gettext.py", line 177, in c2py
result, nexttok = parse(tokenize(plural))
File "/opt/bitnami/python/lib/python2.7/gettext.py", line 114, in _parse
nexttok = next(tokens)
File "/opt/bitnami/python/lib/python2.7/gettext.py", line 91, in _tokenize
raise ValueError('invalid token in plural form: %s' % value)
ValueError: invalid token in plural form: EXPRESSION

jota 2019-01-21 10:56:24 UTC #3

Hi @bi_goutas,

Are you following this guide of our documentation to change the language in edX? Did you install all the required packages and dependencies before running this command?

https://docs.bitnami.com/aws/apps/edx/configuration/change-language/

Thanks

bi_goutas 2019-01-21 12:36:38 UTC #4

hi,

thanks for the replay.
yeah, i followed the bitnami documentation step by step. it worked in the beginning but now when i run this command "sudo /opt/bitnami/apps/edx/bin/paver.edxapp i18n_robot_pull" it always fail. However i18n_generate and i18n_fastgenerate are working.

jota 2019-01-22 09:49:20 UTC #5

Hi @bi_goutas,

Thanks for the information. Our team will investigate this issue and will update this thread as soon as we have more information.

marcos 2019-01-23 13:41:32 UTC #6

Hi @bi_goutas, this is a known issue in edX Hawthorn and was reported some time ago to them: https://openedx.atlassian.net/browse/CRI-129

It seems to have been fixed in the hawthorn.master branch, but there hasn't been a new release yet.

In the meantime you can get past this error by following these steps:

Load the environment:

sudo /opt/bitnami/use_edx
. /opt/bitnami/apps/edx/conf/edxapp_env

Install new releases of Python modules that fix these issues:

sudo pip install git+https://github.com/edx/edx-ora2.git@2.1.18#egg=ora2==2.1.18

Run i18n_robot_pull again (the "--settings" flag is required, we need to update the documentation for it):

sudo /opt/bitnami/apps/edx/bin/paver.edxapp i18n_robot_pull --settings=bitnami

Error with the Forum, comments service

bi_goutas 2019-01-27 10:22:19 UTC #7

Hi @marcos,
It always fail and it gives me the same error .

marcos 2019-01-28 13:38:24 UTC #8

Hi @bi_goutas, I just noticed there is a typo in my previous post.

Instead of running this:

sudo pip install git+https://github.com/edx/edx-ora2.git@2.1.18#egg=ora2==2.1.18

Please run this:

pip install git+https://github.com/edx/edx-ora2.git@2.1.18#egg=ora2==2.1.18

After that proceed with pulling translations as usual:

sudo /opt/bitnami/apps/edx/bin/paver.edxapp i18n_robot_pull --settings=bitnami

You should not get the previous error anymore.

bi_goutas 2019-01-29 10:16:08 UTC #9

hi @marcos,

I've followed the presented steps, however, still it's not working and showing this error

INFO:i18n.generate:Merging djangojs.po locale hi
Traceback (most recent call last):
  File "/opt/bitnami/apps/edx/venvs/edxapp/bin/i18n_tool", line 11, in <module>
    sys.exit(main())
  File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/i18n/main.py", line 60, in main
    return module.main()
  File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/i18n/__init__.py", line 51, in __call__
    return self.run(args)
  File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/i18n/generate.py", line 176, in run
    merge_files(configuration, locale, fail_if_missing=args.strict)
  File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/i18n/generate.py", line 80, in merge_files
    merge(configuration, locale, target, sources, fail_if_missing)
  File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/i18n/generate.py", line 47, in merge
    validate_files(locale_directory, sources)
  File "/opt/bitnami/apps/edx/venvs/edxapp/lib/python2.7/site-packages/i18n/generate.py", line 148, in validate_files
    raise Exception("I18N: Cannot generate because file not found: {0}".format(pathname))
Exception: I18N: Cannot generate because file not found: /opt/bitnami/apps/edx/edx-platform/conf/locale/hi/LC_MESSAGES/djangojs-partial.po


Captured Task Output:
---------------------

---> pavelib.i18n.i18n_robot_pull
---> pavelib.i18n.i18n_clean
git clean -fdX conf/locale
---> pavelib.i18n.i18n_transifex_pull
---> pavelib.i18n.i18n_validate_transifex_config
i18n_tool transifex pull
---> pavelib.i18n.i18n_extract
---> pavelib.prereqs.install_prereqs
---> pavelib.i18n.i18n_validate_gettext
i18n_tool extract
---> pavelib.i18n.i18n_dummy
i18n_tool dummy
i18n_tool generate
python manage.py lms --settings=bitnami compilejsi18n
python manage.py cms --settings=bitnami compilejsi18n
---> pavelib.i18n.i18n_generate_strict
i18n_tool generate --strict

Build failed running pavelib.i18n.i18n_robot_pull: Subprocess return code: 1

marcos 2019-01-29 15:19:54 UTC #10

Hi @bi_goutas, unfortunately it seems https://openedx.atlassian.net/browse/CRI-129 was not resolved yet so you should be seeing that error.

In the meantime while it gets fixed, you can workaround it by changing the following line in /opt/bitnami/apps/edx/edx-platform/pavelib/i18n.py:

    "pavelib.i18n.i18n_generate_strict",

To this:

    "pavelib.i18n.i18n_generate",

After that you should be able to generate locales. Feel free to keep an eye in the page describing the issue, so you can revert that workaround once fix is available.

Hope it finally works for you!

Error with the Forum, comments service

bi_goutas 2019-01-30 14:40:53 UTC #11

thanks @marcos, it works finally.

just to be sure
Validating translations with i18n_tool validate...
i18n_tool validate
Continue with committing these translations (y/n)?

Should I respond with a no, and when I do so, it passes without any error, however, in the Django.po and Djangojs.po files I've found some expressions translated but they don't seem to appear in the web site,For example:
#: lms/templates/student_account/login.underscore
msgid "First time here?"
msgstr "C'est votre première visite ?"

#: lms/templates/student_account/login.underscore
msgid "Create an Account."
msgstr "Créer un compte."

marcos 2019-01-31 09:56:14 UTC #12

Hi @bi_goutas, it could be possible you also need to rebuild assets. You can do this with the command below:

sudo /opt/bitnami/apps/edx/bin/edxapp-migrate-lms
sudo /opt/bitnami/apps/edx/bin/edxapp-migrate-cms

It will take some time. Once it finishes, restart Apache:

sudo /opt/bitnami/ctlscript.sh restart apache

If it works, please let us know.

bi_goutas 2019-01-31 12:14:35 UTC #13

Hi @marcos,
I tried

sudo /opt/bitnami/apps/edx/bin/edxapp-migrate-lms
sudo /opt/bitnami/apps/edx/bin/edxapp-migrate-cms

command and it gives me errors

Running migrations:
  No migrations to apply.
  Your models have changes that are not yet reflected in a migration, and so won't be applied.
  Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
2019-01-31 07:11:34,021 WARNING 12374 [openedx.core.djangoapps.monitoring_utils.middleware] middleware.py:24 - Unable to load NewRelic agent module
2019-01-31 07:11:35,240 INFO 12374 [dd.dogapi] dog_stats_api.py:66 - Initializing dog api to use statsd: localhost, 8125
Operations to perform:
  Apply all migrations: admin, api_admin, assessment, auth, block_structure, bookmarks, catalog, celery_utils, completion, consent, contentserver, contentstore, contenttypes, course_action_state, course_creators, course_groups, course_modes, course_overviews, course_structures, courseware, crawlers, credit, dark_lang, database_fixups, degreed, django_comment_common, django_notify, django_openid_auth, djcelery, edx_oauth2_provider, edx_proctoring, edxval, embargo, enterprise, entitlements, external_auth, integrated_channel, microsite_configuration, milestones, oauth2, oauth2_provider, oauth_provider, organizations, redirects, sap_success_factors, schedules, self_paced, sessions, site_configuration, sites, static_replace, student, submissions, survey, tagging, theming, track, user_api, user_tasks, verified_track_content, verify_student, video_config, video_pipeline, waffle, waffle_utils, wiki, workflow, xblock_config, xblock_django
Running migrations:
  No migrations to apply.
  Your models have changes that are not yet reflected in a migration, and so won't be applied.
  Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.

should I excute a makemigrate?
Where should I do it or what's the equivalent of it in openedx?
Is it because I have changed the /opt/bitnami/apps/edx/edx-platform/pavelib/i18n.py file ?

marcos 2019-01-31 15:09:25 UTC #14

Sorry, I mistakenly posted the wrong commands. Please run these commands instead:

sudo /opt/bitnami/apps/edx/bin/edxapp-update-assets-lms
sudo /opt/bitnami/apps/edx/bin/edxapp-update-assets-cms

bi_goutas 2019-02-03 09:47:59 UTC #15

sorry for the delay,
Finally, It Works!

marcos 2019-02-04 08:15:18 UTC #16

That is awesome! Thansk for letting us know it worked! We will also add a note to our docs that users may need to update assets.

system 2019-02-18 08:15:32 UTC #17

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Bitnami provides free all-in-one installers, virtual machines and cloud images for popular open source applications. Get them now!