ARCAD-Transformer For Unicode

Quite often, software is originally designed to be used in a limited geographical field. However, as the activity changes, software may need to be made usable worldwide.  This leads to the problem of different sets of characters. This problem is even more glaring when the languages are Far Eastern languages that use DBCS (Japan, Korea, Taiwan, China).

It can be tempting to choose the most minimal option in order to provide a limited solution to this problem. Yet this requires making code more complex in certain places, without providing an overall solution to this issue.

THE SOLUTION

The Unicode approach is entirely different. A choice is made once and for all to integrate all of these language constraints into the software by:

• putting all of the database alphanumeric fields into Unicode
• putting all of the management rules into Unicode

But the EBCDIC (SBCS/DBCS) must be conserved when necessary, as certain functionalities don’t support Unicode, or support it badly:

• DSPF/PRTF
• CLP/CLLE variables
• Using commands
• API calls from the OS
• Others

These limits are the stumbling blocks of the “All Unicode” solution, as they require managing despite these many conversions, by polluting the source code kept in place.

Note: It is sometimes necessary to enlarge certain fields if we want DCBS characters to be accepted in them (minimum length = 4).

THE ARCAD FOR UNICODE SOLUTION

This is why the ARCAD for Unicode method was designed to take these various points into account, with the objective of:

• conserving maintained source code that is not overloaded (very similar to the old source code)
• inserting the necessary conversions upon compilation
• optimizing conversions of literals.

There are two steps in the solution:

1/ARCAD-Transformer for Unicode. This involves a one-shot transformation of the application components into Unicode via ARCAD-Transformer:

• Assistance in selecting the fields concerned
  • only from database fields
  • or for all alphanumeric fields
• Propagation through? many iterations
  • warning reports that flag cases that require sample manual adaptations
  • redefining numeric fields into alphabetical fields that become Unicode
  • transition of the indicator parameter to Unicode
  • using instructions that do not support Unicode
  • detecting literals with variant characters or characters with no equivalent in certain languages
• Transforming source code into Unicode
  • Transitioning database fields into Unicode UCS2 (Graphic CCSID 13488)
  • Changing RPGLE fields into type C, or expanding structures (certain parts of which become Unicode)
  • Declaring fields in Specification D instead of Specification C
  • Doubling lengths in CLP/CLLE fields
  • Converting operation code that does not support Unicode into operation code that does allow it
  • Changing literals into Unicode constants
• Generating the recovery program (between the old SBCS files and the new Unicode files).

2/ ARCAD Pre-compiler for Unicode: With the aim of NOT MAKING THE SOURCE CODE RETAINED MORE COMPLEX, this code – modified during the transition to Unicode, has no more lines than the old source code and the conversions to SBCS fields that are necessary for certain functionalities do not appear in it.

Therefore, compilation of this source code (for example: RPGLE type) goes through the ARCAD Pre-compiler for Unicode, which is in charge of:

• Controlling certain instructions (e.g., using variant characters or characters that do not exist in every country)
• Ensuring pre-display/post-entry conversions for DSPF/PRTF fields
• Making CLP/CLLE work as a language that supports Unicode (for literals, concatenations, command calls, etc.) by inserting all of the necessary conversions
• In RPGLE, changing all of the literals into Unicode constants, in order to optimize processing time
• Inserting pre-/post non-Unicode third party program call conversions (other application or OS API).
• And more…

A second piece of source code is thus generated (e.g., RPGLE2 type) and it is this code that is actually compiled.

If you develop via the LPEZ publisher in R.D.P., pre-compilation errors (ARCAD) or compilation errors (IBM) are brought up to the level of the corresponding line within the source code that is kept.

ARCAD-Skipper ensures coherence management of these two kinds of source code, to which only one object corresponds. You modify only the source code maintained; the source code generated is retained and regenerated as many times as you like. It should be noted that an ARCAD snap-in to the IBM Debugger allows you to view/modify CLP fields as if they were Unicode type.

Various approaches

The Unicode conversion can concern all of the fields of all files (and processing). The solution also lets you be more selective, only allowing Unicode for certain application entities. In this case, you can use another ARCAD-Transformer functionality to perform a guided renaming of the fields of:

• all files
• programs and screen

The standardized fields can intrinsically contain the management rule concerning whether they can support Unicode or not.