Migração do OTRS / ((OTRS)) Community Edition versão 6 para OTOBO versão 10

Bem-vindo e obrigado por escolher o OTOBO!

OTRS, ((OTRS)) Community Edition e OTOBO são muito abrangentes e flexíveis em sua aplicação. Portanto, toda migração para o OTOBO requer uma preparação completa e, possivelmente, algum retrabalho também.

Reserve um tempo para a migração e siga estas instruções passo a passo.

If you have any problem or question, please do not despair. Call our support line, write an email, or post your query in the OTOBO Community forum at https://forum.otobo.org/. We will find a way to help you!

Nota

After the migration the data previously available in OTRS 6 will be available in OTOBO 10. We do not modify any data of the OTRS 6 installation during the migration.

Overview over the Supported Migration Szenarios

With the OTOBO Migration Interface it is possible to employ the following migration strategies:

  1. The general migration strategy.

    This is the regular way to perform a migration. Many different different combinations are supported:

    Change server:

    Migrate and simultaneously move to a new application server.

    Separate application and web servers:

    It’s your choice whether you want to run application and database server on the same host or each on a dedictated host. This choice is regardless of the previous setup in OTRS / ((OTRS)) Community Edition.

    Different databases:

    Migrate from any of the supported databases to any other supported database.

    Different operating system:

    Switch from any supported operating system to any other supported operating system.

    Docker:

    Migrate to a Docker-based installation of OTOBO 10.

  2. A variant of the general strategy where the database migration is streamlined.

    Use the ETL-like migration when the source database mustn’t suffer from increased load or when access to the source database is a bottleneck. In the general strategy, the data is row by row first read from the otrs database and then inserted into the OTOBO database. In this variant, the complete otrs database tables are first exported, then transformed, and then imported into the otobo database.

  3. Migration from an Oracle based OTRS 6 installation to an Oracle based OTOBO installation.

    This is a special case that is not supported by the general migration strategy. This means that a variant of the streamlined strategy must be used.

Aviso

All strategies work for both Docker-based and native installations. But for Docker-based installations some peculiarities have to be considered. These peculiarities are handled in the optional steps.

Nota

It is also feasible to clone the OTRS datase to the OTOBO database server before the actual migration. This can speed up the general migration strategy.

Requerimentos de migração

  1. Basic requirement for a migration is that you already have an ((OTRS)) Community Edition or OTRS 6.0.* running, and that you want to transfer both configuration and data to OTOBO.

Aviso

Please consider carefully whether you really need the data and configuration. Experience shows that quite often a new start is the better option. This is because in many cases the previously used installation and configuration was rather suboptimal anyways. It might also make sense to only transfer the ticket data and to change the basic configuration to OTOBO Best Practice. We are happy to advise you, please get in touch at hello@otobo.de or ask your question in the OTOBO Community forum at https://forum.otobo.org/.

  1. Você precisa de uma instalação OTOBO em execução para iniciar a migração a partir daí!
  2. Essa instalação do OTOBO deve conter todos os pacotes OPM instalados no seu OTRS que você também deseja usar no OTOBO.
  3. If you are planning to migrate to another server, then the OTOBO webserver must be able to access the location where your ((OTRS)) Community Edition or OTRS 6.0.* is installed. In most cases, this is the directory /opt/otrs on the server running OTRS. The read access can be effected via SSH or via file system mounts.
  4. The otrs database must be accessible from the server running OTOBO. Readonly access must be granted for external hosts. If access is not possible, or when the speed of the migration should be optimised, then a dump of the database is sufficient.

Nota

Se o acesso SSH e ao banco de dados entre ambos servidores não for possível, migre o OTRS para o OTOBO no mesmo servidor e somente então mova a nova instalação.

Passo 1: Instalação do novo Sistema OTOBO

Please start with installing a new OTOBO system. Your old OTRS / ((OTRS)) Community Edition installation will be migrated to that new system. We strongly recommend to read the chapter Instalação do OTOBO. For Docker-based installations we refer to the chapter Instalando usando Docker e Docker Compose.

Aviso

Under Apache, there are pitfalls with running two independent mod_perl applications under on the same webserver. Therefore, it is advised to run OTRS and OTOBO on separate webservers. Alternatively remove the OTRS configuration from Apache before installing OTOBO. Use the command a2query -s and check the directories /etc/apache2/sites-available and /etc/apache2/sites-enabled for inspecting which configurations are currently available and which are enabled.

After finishing the installation please log in as root@localhost. Navigate to the OTOBO Admin Area Admin -> Packages and install all required OTOBO OPM packages.

Os seguintes pacotes OPM e OTRS “Feature Addons” NÃO precisa e NÃO deve ser instalado, pois esses recursos já estão disponíveis no padrão OTOBO:
  • OTRSHideShowDynamicField
  • RotherOSSHideShowDynamicField
  • TicketForms
  • RotherOSS-LongEscalationPerformanceBoost
  • Znuny4OTRS-AdvancedDynamicFields
  • Znuny4OTRS-AutoSelect
  • Znuny4OTRS-EscalationSuspend
  • OTRSEscalationSuspend
  • OTRSDynamicFieldDatabase
  • OTRSDynamicFieldWebService
  • OTRSBruteForceAttackProtection
  • Znuny4OTRS-ExternalURLJump
  • Znuny4OTRS-QuickClose
  • Znuny4OTRS-AutoCheckbox
  • OTRSSystemConfigurationHistory
  • Znuny4OTRS-PasswordPolicy

Step 2: Deactivate SecureMode on OTOBO

After installing OTOBO, please log in again to the OTOBO Admin Area Admin -> System Configuration and deactivate the config option SecureMode.

Nota

Do not forget to actually deploy the changed setting.

Step 3: Stop the OTOBO Daemon

This is necessary when the OTOBO Daemon is actually running. Stopping the Daemon is different between Docker-based and non-Docker-based installations.

In the non-Docker case execute the following commands as the user otobo:

# in case you are logged in as root
root> su - otobo

otobo> /opt/otobo/bin/Cron.sh stop
otobo> /opt/otobo/bin/otobo.Daemon.pl stop --force

When OTOBO is running in Docker, you just need to stop the service daemon:

docker_admin> cd /opt/otobo-docker
docker_admin> docker-compose stop daemon
docker_admin> docker-compose ps     # otobo_daemon_1 should have exited with the code 0

Nota

É recomendável executar um backup de todo o sistema OTOBO neste momento. Se algo der errado durante a migração, você não precisará repetir todo o processo de instalação, mas poderá importar o backup para uma nova migração.

Ver também

Nós aconselhamos a ler o capitulo OTOBO Backup e Restauração.

Optional Step: Mount /opt/otrs for Convenient Access

Often OTOBO should be running on a new server where /opt/otrs isn’t available initially. In these cases the directory /opt/otrs on the OTRS server can be mounted into the file system of the OTOBO server. When a regular network mount is not possible, then using sshfs might be an option.

Optional Step: Install sshpass and rsync when /opt/otrs Should be Copied via ssh

This step is only necessary when you want to migrate OTRS from another server and when /opt/otrs from the remote server hasn’t been mounted on the server running OTOBO.

The tools sshpass and rsync are needed so that migration.pl can copy files via ssh. For installing sshpass, please log in on the server as user root and execute one of the following commands:

$ # Install sshpass under Debian / Ubuntu Linux
$ sudo apt-get install sshpass
$ #Install sshpass under RHEL/CentOS Linux
$ sudo yum install sshpass
$ # Install sshpass under Fedora
$ sudo dnf install sshpass
$ # Install sshpass under OpenSUSE Linux
$ sudo zypper install sshpass

O mesmo deve ser feito para rsync quando ainda não está disponível.

Step 4: Preparing the OTRS / ((OTRS)) Community Edition system

Nota

Certifique-se de ter um backup válido do seu sistema OTRS / ((OTRS)) Community Edition também. Sim, não tocamos em nenhum dado OTRS durante a migração, mas às vezes uma entrada incorreta é suficiente para causar problemas.

Agora estamos prontos para a migração. Antes de tudo, precisamos garantir que nenhum ticket seja processado e que nenhum usuário efetue logon no OTRS:

Por favor, faça o login na área de administração OTRS Admin ->  System Maintenance e adicione um novo slot de manutenção do sistema por algumas horas. Depois disso, exclua todas as sessões de agente e usuário (Admin ->  Sessions) e faça logout.

Interromper todos os serviços relevantes e o Daemon OTRS

Certifique-se de que não haja serviços ou cron jobs em execução.

root> su - otrs
otrs>
otrs> /opt/otrs/bin/Cron.sh stop
otrs> /opt/otrs/bin/otrs.Daemon.pl stop --force
otrs> /opt/otrs/bin/otrs.Console.pl Maint::Cache::Delete
otrs> /opt/otrs/bin/otrs.Console.pl Maint::Session::DeleteAll
otrs> /opt/otrs/bin/otrs.Console.pl Maint::Loader::CacheCleanup
otrs> /opt/otrs/bin/otrs.Console.pl Maint::WebUploadCache::Cleanup

Optional Step for Docker: make required data available inside container

Existem alguns detalhes a serem considerados quando a instalação do OTOBO está em execução no Docker. O mais relevante: os processos em execução em um contêiner Docker geralmente não podem acessar diretórios fora do contêiner. Porém, há uma exceção: os diretórios montados como volumes no contêiner podem ser acessados. Além disso, observe que o banco de dados MariaDB rodando em otobo_db_1 não é diretamente acessível de fora da rede de contêiner.

Nota

Nos comandos de amostra, presumimos que o usuário docker_admin é usado para interagir com o Docker. O administrador do Docker pode ser o usuário root do host do Docker ou um usuário dedicado com as permissões necessárias.

Copie /opt/otrs dentro do volume otobo_opt_otobo

Nesta seção, presumimos que o diretório inicial OTRS /opt/otrs está disponível no host Docker.

Existem pelo menos duas possibilidades viáveis:

  1. copie /opt/otrs para o volume existente otobo_opt_otobo
  2. montar /opt/otrs como um volume adicional

Vamos nos concentrar na opção a. aqui.

Primeiro, precisamos descobrir onde o volume otobo_opt_otobo está disponível no host Docker.

docker_admin> otobo_opt_otobo_mp=$(docker volume inspect --format '{{ .Mountpoint }}' otobo_opt_otobo)
docker_admin> echo $otobo_opt_otobo_mp  # just a sanity check

Para uma cópia segura, usamos rsync. Dependendo da configuração do Docker, o comando rsync pode precisar ser executado com sudo.

docker_admin> # when docker_admin is root
docker_admin> rsync --recursive --safe-links --owner --group --chown 1000:1000 --perms --chmod "a-wx,Fu+r,Du+rx" /opt/otrs/ $otobo_opt_otobo_mp/var/tmp/copied_otrs
docker_admin> ls -la $otobo_opt_otobo_mp/var/tmp/copied_otrs  # just a sanity check

docker_admin> # when docker_admin is not root
docker_admin> sudo rsync --recursive --safe-links --owner --group --chown 1000:1000 --perms --chmod "a-wx,Fu+r,Du+rx" /opt/otrs/ $otobo_opt_otobo_mp/var/tmp/copied_otrs
docker_admin> sudo ls -la $otobo_opt_otobo_mp/var/tmp/copied_otrs  # just a sanity check

Este diretório copiado estará disponível como /opt/otobo/var/tmp/copied_otrs dentro do contêiner.

Optional Step: Streamlined migration of the database

In the general migration strategy, all data in the database tables is copied row by row from the OTRS database into the OTOBO database. Exporting the data from the OTRS database and importing it into the OTOBO database might save time and is more stable in some circumstances.

Nota

This variant works for both Docker-based and native installations.

Nota

These instructions assume that OTRS is using MySQL as its backend.

First of all, we need a dump of the needed OTRS database tables. Then we need to perform a couple of transformations:

  • convert the character set to utf8mb4
  • rename a couple of tables
  • shorten some table columns

After the transfomation we can overwrite the tables in the OTOBO schema with the transformed data from OTRS. Effectively we need not a single dump file, but several SQL scripts.

Quando o mysqldump é instalado e uma conexão com o banco de dados OTRS é possível, você pode criar o dump do banco de dados diretamente no host Docker. Este caso é suportado pelo script bin/backup.pl.

Aviso

Isso requer que uma instalação OTOBO esteja disponível no host Docker.

otobo> cd /opt/otobo
otobo> scripts/backup.pl -t migratefromotrs --db-name otrs --db-host=127.0.0.1 --db-user otrs --db-password "secret_otrs_password"

Nota

Alternatively, the database can be dumped on another server and then be transferred to the Docker host afterwards. An easy way to do this is to copy /opt/otobo to the server running OTRS and perform the same command as above.

The script bin/backup.pl generates four SQL scripts in a dump directory, e.g. in 2021-04-13_12-13-04 In order to execute the SQL scripts, we need to run the command mysql.

Native installation:

otobo> cd <dump_dir>
otobo> mysql -u root -p<root_secret> otobo < otrs_pre.sql
otobo> mysql -u root -p<root_secret> otobo < otrs_schema_for_otobo.sql
otobo> mysql -u root -p<root_secret> otobo < otrs_data.sql
otobo> mysql -u root -p<root_secret> otobo < otrs_post.sql

Docker-based installation:

Run the command mysql within the Docker container db for importing the database dump files. Note that the password for the database root is now the password that has been set up in the file .env on the Docker host.

docker_admin> cd /opt/otobo-docker
docker_admin> cd <dump_dir>
docker_admin> docker-compose exec -T db mysql -u root -p<root_secret> otobo < <dump_dir>/otrs_pre.sql
docker_admin> docker-compose exec -T db mysql -u root -p<root_secret> otobo < <dump_dir>/otrs_schema_for_otobo.sql
docker_admin> docker-compose exec -T db mysql -u root -p<root_secret> otobo < <dump_dir>/otrs_post.sql
docker_admin> docker-compose exec -T db mysql -u root -p<root_secret> otobo < <dump_dir>/otrs_data.sql

Para uma verificação rápida se a importação funcionou, você pode executar os seguintes comandos.

otobo> mysql -u root -p<root_secret> -e 'SHOW DATABASES'
otobo> mysql -u root -p<root_secret> otobo -e 'SHOW TABLES'
otobo> mysql -u root -p<root_secret> otobo -e 'SHOW CREATE TABLE ticket'

or

docker_admin> docker-compose exec -T db mysql -u root -p<root_secret> -e 'SHOW DATABASES'
docker_admin> docker-compose exec -T db mysql -u root -p<root_secret> otobo -e 'SHOW TABLES'
docker_admin> docker-compose exec -T db mysql -u root -p<root_secret> otobo -e 'SHOW CREATE TABLE ticket'

The database is now migrated. This means that during the next step we can skip the database migration. Watch out for the relevant checkbox.

Step 5: Perform the Migration!

Please use the web migration tool at http://localhost/otobo/migration.pl. Be aware that you might have to replace “localhost” with your OTOBO hostname and you might have to add your non-standard port. The application then guides you through the migration process.

Aviso

Às vezes, é mostrado um aviso de que a desativação do SecureMode não foi detectada. Reinicie o servidor da web neste caso. Isso força o servidor web a ler a configuração atual.

# native installation
root> service apache2 restart

# Docker-based installation
docker_admin> cd /opt/otobo-docker
docker_admin> docker-compose restart web
docker_admin> docker-compose ps     # otobo_web_1 should be running again

Nota

If OTOBO runs inside a Docker container, keep the default settings localhost for the OTRS server and /opt/otobo/var/tmp/copied_otrs for the OTRS home directory. This is the path of the data that was copied in the optional step.

Nota

Os valores padrão para usuário e senha do banco de dados OTRS são obtidos de Kernel/Config.pm no diretório inicial do OTRS. Altere as configurações propostas se estiver usando um usuário de banco de dados dedicado para a migração. Altere também as configurações ao trabalhar com um banco de dados que foi copiado para o contêiner otobo_db_1 do Docker.

Nota

No caso do Docker, um banco de dados em execução no host Docker não será acessível via 127.0.0.1 de dentro do contêiner Docker. Isso significa que a configuração 127.0.0.1 não será válida para o campo de entrada OTRS Server. Nesse caso, insira um dos endereços IP alternativos relatados pelo comando hostname --all-ip-addresses para o OTRS Server.

Nota

Ao migrar para um novo servidor de aplicativos ou para uma instalação baseada em Docker, muitas vezes o banco de dados não pode ser acessado a partir da instalação de destino. Isso geralmente se deve ao fato de que o usuário do banco de dados otobo só pode se conectar a partir do host em que o banco de dados é executado. Para permitir o acesso de qualquer maneira, é recomendável criar um usuário de banco de dados dedicado para a migração. Por exemplo. CREATE USER 'otrs_migration'@'%' IDENTIFIED BY 'otrs_migration'; e GRANT SELECT, SHOW VIEW ON otrs.* TO 'otrs_migration'@'%';. Este usuário pode ser descartado novamente após a migração: DROP USER 'otrs_migration'@'%'.

Custom settings in Kernel/Config.pm are carried over from the old OTRS installation to the new OTOBO installation. When you have custom settings, then please take a look at the migrated file /opt/otobo/Kernel/Config.pm. You might want to adapt custom pathes or LDAP settings. In the best case you might find that some custom setting are longer needed.

Quando a migração estiver concluída, não se apresse e teste todo o sistema. Depois de decidir que a migração foi bem-sucedida e que você deseja usar o OTOBO a partir de agora, inicie o Daemon OTOBO:

root> su - otobo
otobo>
otobo> /opt/otobo/bin/Cron.sh start
otobo> /opt/otobo/bin/otobo.Daemon.pl start

No caso do Docker:

docker_admin> cd ~/otobo-docker
docker_admin> docker-compose start daemon

Step 6: After Successful Migration!

  1. Desinstale sshpass se você não precisar mais.
  2. Drop the databases and database users dedicated to the migration if you created any.
  3. Divirta-se com o OTOBO!

Known Migration Problems

1. Login after migration not possible

Durante nossos testes de migração, o navegador usado para a migração às vezes apresentava problemas. Depois de reiniciar o navegador, esse problema geralmente era resolvido. Com o Safari, às vezes era necessário excluir manualmente a sessão antiga do OTRS.

2. Final page of the migration has a strange layout due to missing CSS files

Isso pode acontecer quando a configuração ScriptAlias tem um valor não padrão. A migração simplesmente substitui otrs por otobo. Isso pode fazer com que o CSS e o JavaScript não possam mais ser recuperados no OTOBO. Quando isso acontecer, verifique as configurações em Kernel/Config.pm e reverta-as para os valores normais.

3. Migration stops due to MySQL errors

On systems that experienced problems with an upgrade in the past, the migration process may stop due to MySQL errors in the tables ticket and ticket_history. Usually these errors are NULL values in the source table that are no longer allowed in the target table. These conflicts have to be manually resolved before you can resume the migration.

As of OTOBO 10.0.12 there is a check in migration.pl that checks for NULL values before the data transfer is done. Note, that the resolution still needs to be performed manually.

4. Errors in Step 5 when migrating to PostgreSQL

In these cases the not so helpful message “System was unable to complete data transfer.” is shown by migration.pl. The Apache logfile, and the OTOBO logfile, show a more meaningful message: “Message: ERROR: permission denied to set parameter “session_replication_role”, SQL: ‘set session_replication_role to replica;’”. In order to give the database user otobo the needed superuser privileges, run the following statement as the PostgreSQL admin: ALTER USER otobo WITH SUPERUSER;. Then retry running http://localhost/otobo/migration.pl. After the migration, return to the normal state by running ALTER USER otobo WITH NOSUPERUSER.

It is not clear yet, whether the extended privileges have to be granted in every setup.

5. Problems with the Deployment the Merged System Configuration

The system configuration is migrated after the database tables were migrated. In this context, migration means merging the default settings of OTOBO with the system configuration of the source OTRS system. Inconsistencies can arise in this step. An real life example is the setting Ticket::Frontend::AgentTicketQuickClose###State. This setting is new in OTOBO 10 and the default value is the state closed successful. But this setting is invalid when the state closed successful has been dropped or renamed in the source system. This inconsistency is detected as an error in the migration step Migrate configuration settings. Actually, the merged system configuration is stored in the database, but additional validity checks are performed during deployment.

The problem must be alleviated manually by using OTOBO console commands.

  • List the inconsistencies with the command bin/otobo.Console.pl Admin::Config::ListInvalid
  • Interactively fix the invalid values with bin/otobo.Console.pl Admin::Config::FixInvalid
  • Deploy the collected changes from migration.pl, including the deactivated SecureMode with bin/otobo.Console.pl Maint::Config::Rebuild

After these manual steps you should be able to run migration.pl again. The migration will continue with the step where the error occurred.

Etapa 7: Tarefas de migração manual e alterações

1. Password policy rules

Com o OTOBO 10, uma nova política de senha padrão para usuários de agentes e clientes entra em vigor, se a autenticação local for usada. As regras da política de senha podem ser alteradas na configuração do sistema (PreferencesGroups###Password and CustomerPersonalPreference###Password).

Regra de política de senha Padrão
PasswordMinSize 8
PasswordMin2Lower2UpperCharacters Sim
PasswordNeedDigit Sim
PasswordHistory 10
PasswordTTL 30 dias
PasswordWarnBeforeExpiry 5 dias
PasswordChangeAfterFirstLogin Sim

2. Under Docker: Manually migrate cron jobs

Em uma instalação não Docker do OTOBO, há pelo menos um cron job que verifica a integridade do Daemon. No Docker, esse cron job não existe mais. Além disso, não há daemon cron em execução em nenhum dos contêineres do Docker. Isso significa que você deve procurar uma solução individual para sistemas OTRS com cron jobs específicos do cliente (por exemplo, backup do banco de dados).

Special topics

Migration from Oracle to Oracle

For migration to Oracle the ETL-like strategy must be employed. This is because Oracle provides no easy way to temporarily turn off foreign key checks.

On the OTOBO host a Oracle client and the Perl module DBD::Oracle must be installed.

Nota

When using the Oracle instant client, then the optional SDK is also needed for installing DBD::Oracle.

There are many ways of cloning a schema. In the sample commands we use expdb and impdb which use Data Pump under the hood.

Nota

The connect strings shown in this documentation refer to the case when both source and target database run in a Docker container. See also https://github.com/bschmalhofer/otobo-ideas/blob/master/oracle.md .

  1. Clear out otobo

Stop the webserver for otobo, so that the DB connection for otobo is closed.

-- in the OTOBO database
DROP USER otobo CASCADE
  1. Export the complete OTRS schema.
mkdir /tmp/otrs_dump_dir
-- in the OTRS database
CREATE DIRECTORY OTRS_DUMP_DIR AS '/tmp/otrs_dump_dir';
GRANT READ, WRITE ON DIRECTORY OTRS_DUMP_DIR TO sys;
expdp \"sys/Oradoc_db1@//127.0.0.1/orclpdb1.localdomain as sysdba\" schemas=otrs directory=OTRS_DUMP_DIR dumpfile=otrs.dmp logfile=expdpotrs.log
  1. Import the OTRS schema, renaming the schema to ‘otobo’.
impdp \"sys/Oradoc_db1@//127.0.0.1/orclpdb1.localdomain as sysdba\" directory=OTRS_DUMP_DIR dumpfile=otrs.dmp logfile=impdpotobo.log remap_schema=otrs:otobo
-- in the OTOBO database
-- double check
select owner, table_name from all_tables where table_name like 'ARTICLE_DATA_OT%_CHAT';

-- optionally, set the password for the user otobo
    ALTER USER otobo IDENTIFIED BY XXXXXX;
  1. Adapt the cloned schema otobo
cd /opt/otobo
scripts/backup.pl --backup-type migratefromotrs # it's OK that the command knows only about the otobo database, only last line is relevant
sqlplus otobo/otobo@//127.0.0.1/orclpdb1.localdomain < /home/bernhard/devel/OTOBO/otobo/2021-03-31_13-36-55/orclpdb1.localdomain_post.sql >sqlplus.out 2>&1
double check with `select owner, table_name from all_tables where table_name like 'ARTICLE_DATA_OT%_CHAT';
  1. Start the web server for otobo again
  2. Proceed with step 5, that is with running migration.pl.