Mass data import

There was an issue with importing all UK post codes from a csv file, about 3m post codes. Importing using rapidstart services (excel import) can cause buffer overflow messages. excel itself has row/size limitations. Increasing MaxNoOfXMLRecordsToSend in config file ClientUserSettings.config from default value 5000 to e.g. 20000 is no problem and can help. Also changing MaxUploadSize in server config file CustomSettings.config is an option (also available via nav service admin console). Better choice for mass data import are dataports (older nav versions) and xmlports.

Another option is to develope a report, which imports the file contents and loops through the lines. quite simple, no memory issues.

create a new report, add following code to trigger OnPreReport():

OnPreReport()
// variables
 PostCode, Record, Post Code
 file, File
 fileName, Text, 250
 line, Text, 1024
 dlg, Dialog
 idx, Integer
 txtValue, Text, 100
// code
 PostCode.DELETEALL(FALSE);
 COMMIT;
 dlg.OPEN('#1###### #2######'); // show progress dialog
 idx := 1;

// downloaded post code file from https://www.doogal.co.uk/PostcodeDownloads.php
// as test file, size: 500k, 2.1m lines, some of them contain obsolete post codes
 fileName := 'c:\temp\England postcodes.csv';

 file.WRITEMODE := FALSE;
 file.TEXTMODE := TRUE;
 file.OPEN(fileName);
 file.READ(line); // skip header line
 WHILE file.READ(line) > 0 DO BEGIN
   // skip obsolete post codes: 2. value = No
   IF SELECTSTR(2,line) = 'Yes' THEN BEGIN 
     PostCode.Code := SELECTSTR(1,line);
     PostCode.VALIDATE(City,GetValue(SELECTSTR(15,line)));
     PostCode.County := GetValue(SELECTSTR(8,line));
     PostCode."Country/Region Code" := 'GB';
     PostCode.INSERT;
     dlg.UPDATE(1,idx);
     dlg.UPDATE(2,PostCode.Code);
     idx += 1;
   END;
 END;

 file.CLOSE;
 dlg.CLOSE;

 MESSAGE(FORMAT(idx) + ' post codes imported.');

LOCAL GetValue(txtValue : Text[100]) : Text
 txtValue := DELCHR(txtValue,'<','"'); // remove leading "  
 txtValue := DELCHR(txtValue,'>','"'); // remove trailing "
 IF STRLEN(txtValue) > 30 THEN // fields City,County are Text[30]
   txtValue := COPYSTR(txtValue,1,30); // cut text, leading 30 chars

 EXIT(txtValue);

report runs with 1.5m valid lines/records about 5 min.

cheers

Advertisements

Textual Data Export by Configuration

An often by customers wanted feature is to get a data export to a text/csv file, which can then be edited in excel or text editor.

Yes, you can use Rapidstart Services. But with that feature you can only export data in excel format, not in text format. That’s ok for editing in Excel and reimport the changed data back to NAV. For data exchange scenarios you always need text format. An other point is, that it’s quite often said, Rapidstart Services are not that easy to use for end users and has a couple of bugs. I also read quite often that this feature is not recommended at all. So what else can a customer do? Contact the NAV partner, who then shall develope a new xmlport, report or codeunit to do that job. That’s the usual way.

The german localised version of Dynamics Nav contains a really nice feature, simply called “Data Exports”. You can find it in menu /Departments/Administration/Application Setup/Financial Management/General/Data Exports. This feature is mainly used to export business data for auditing purposes according the GDPdU (Process for data access and testability of digital documents).

With the feature “Data Exports” there is an additional way for these kind of issues, it is possible to export data to csv files without developing! Great thing. So let’s have a look, how we can use that.

dx1

First we need at least one definition group. Then we define the according record definition (Button “Record Definitions”).

dx2

Here we need values for Code, Description and Export Path. In that case i want to export Sales Orders. “DTD File Name” is a mandatory field. I checked the code. The file is not processed, using different kinds of dtd files does not change anything in the resulting export files. The name of the dtd file is simply written to the file index.xml, which contains the datastructure. To get the thing run we create an empty text file, call it empty.dtd and add only one line of text:

Save the text file and click on Button Import in menu tab “DTD File”, select file empty.dtd and click ok. After that the file is imported and the file’s name is set to column “DTD File Name”.

Now let’s define the table and the fields. For that select the record definition and click on “Record Source”.

dx3

In the header area we set table no. to 36 (Sales Header) and the Key No. to 1 (Primary Key). Optional you can set the period field no., here to 19 (Order Date). In Column Table Filter you can set filters, if you do not want to export all records. Here i set the field to “Sales Header: Document Type=Order”, because i only want to export Sales Orders.

In the Fields Area we add the export fields by clicking on the Add Button. The key fields should be in first place. With MoveUp and MoveDown you can change the position of the fields. That’s all.

You can, if you wish, add more tables in the header area, you can set relationships, e.g. between Sales Header and Sales Line. But for now let’s run the thing. First we click Validate and get “The data export record source validated correctly.”, means all is ok.

So, after closing the page “Data Export Records Source” we are back in page “Data Export Records Definitions”. Here we click on Button Export in menu tab “Process”. Report 11015 “Export Business Data” is started:

dx4

Here set start and end date. These fields are mandatory and are applied to the period field of the header line, in that case to field “Order Date”. After execution we get some files in the export path, subfolder SALESORDER.

dx5

File empty.dtd is simply copied, file index.xml contains the data structure:

dx6

and file SalesHeader.txt contains the exported data (csv format):

dx7

There we are!
We got a csv export file without any development, only by configuration!

This can, as told before, extended by adding more lines in the header area of page “record source”:

dx8

Here you can see 2 header lines “Sales Header” and “Sales Line”, which is indented and has a defined relationship shown in page relationship in the bottom right corner of the above screenshot. In that case you get 2 export files, one per table.

Important: If you are interested in that feature, you could download the german localised version. But … for that you need the according license!
Exporting the nav objects as text export and renumbering them to the 50000s object range is possible, but illegal! So if you like that kind of feature, try to redevelope that functionality. Do not copy these objects to your database, do not reuse the code. If you want to do that, please contact your Nav partner or Microsoft for license clarifcation. You could suggest to add that functionality to the W1 version.

For more details about that feature follow this.

cheers

Setting up precisions

In Dynamics Nav amount and currency precisions are set on a central point, in the General Ledger Setup. These settings apply for the LCY (Local currency). Foreign currencies are configured in page Currency Card (on base of table Currency).

gl-setup

Fields “Amount Decimal Places”, “Amount Rounding Precision”, “Unit-Amount Decimal Places” and “Unit-Amount Rounding Precision” are maybe missing in this page. In the standard page they are not shown. In that case adde them to the page in the development environment.

curr-setup

In the currency card you can configure foreign currencies. Here you find the same fields to set the precision as in the general ledger setup. This is also the place to set/update the exchange rates.

For details about the fields in G/L Setup follow
https://msdn.microsoft.com/en-us/library/hh168649(v=nav.70).aspx
https://msdn.microsoft.com/en-us/library/hh169567(v=nav.70).aspx

For details about the fields in the Currency Card follow
https://msdn.microsoft.com/en-us/library/hh169173(v=nav.70).aspx
https://msdn.microsoft.com/en-us/library/hh170490(v=nav.70).aspx
https://msdn.microsoft.com/en-us/library/hh168784(v=nav.70).aspx

Additional it’s possible to activate automatic invoice rounding:
To make sure that sales and purchase invoices are rounded automatically, you must activate the invoice rounding function. In the Search box, enter Sales & Receivables Setup or Purchases & Payables Setup, open the page, on the General FastTab, select the Invoice Rounding check box. You can activate invoice rounding separately for sales and purchase invoices.

cheers

 

Localisations for NAV

Localizations for specific countries are quite often an issue. Microsoft provides only a couple of localizations: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Great Britain, Iceland, India*, Italy, Mexico, Netherlands, New Zealand, Norway, Russia*, Spain, Sweden, Switzerland, United States.

* The supported countries can differ between the Nav versions.

The complete lists you’ll find here.

For countries, where there is no localization, best start with Dynamics Nav W1 version. This is the base version of Dynamics Nav, contains no localization at all. So it can be used in every country, but needs a couple of customizations according to national laws, e.g. changes in the accounting/finance area, new and customized reports a.s.o. Additional you’ll may need a new language layer. For that look for a Nav partner located in your region, start with Pinpoint.

Following a list of Nav Partners, who have developed additional localizations. This list will be periodically updated:

Europe:

  • Slovenian Nav Partner Business Solutions d.o.o., also known for their great Permission & Role Wizard for Nav 2009, has developed Nav 2013 localizations for following countries: Slovenia, Croatia, Serbia, Bosnia and Herzegovina, Montenegro, Macedonia, Kosovo. Included are additional functionalities and tools. They provide the localizations in 3 packages: basic, extended, integrated. Maybe they also have developed localizations for the newer Nav versions. If you need it, contact them directly.
  • Adacta group: localization for Slovenia, Croatia, Serbia, Macedonia and Montenegro.
  • Effekt: located in Athens, developed greek localization at least for Nav 2013.
  • IT.integro: localization for Poland. thx to daniele rebussi for the hint.
  • Awara IT Solutions: localization for Russia, especially for the accounting/finance area.
  • Multisoft: localization for Hungary.
  • Alna: localization for Lithuania. Alna is active in Lithuania, Latvia and Poland.
  • EDPA: localization for Latvia
  • CDL.System: Czech localization
  • Autocont CZ a.s.: localizations for Czech Republic and Slovakia
  • Wbi: localization for Slovakia
  • Intelligent Systems: localization for Bulgaria
  • Team-Vision: localization for Bulgaria
  • Infotekas: localization for Turkey. Nav versions 2013 R2, 2015, 2016 in development.
  • Softstore / Link Consulting: localization for Portugal

Americas:

  • Cadia Consulting, located in Sao Paolo, developed a brazilian localization.
  • Master Consulting Group: localizations for LATAM, Peru, Ecuador, Colombia and Bolivia. They are also developing localizations for Chile and Panama.
  • Voxa: localization for Argentina
  • Grvppe: localization for Brazil

Asia:

  • PT SME Solution, located in Jakarta, Indonesia, developed an indonesian localization at least for Nav 2013.
  • PBC and Tectura Korea: localization for China.
  • PBC also developed a localization for Japan.
  • Ibizcs: localizations for Singapore, Malaysia, Thailand, Indonesia and China. Provided Nav versions: Nav 2009, 2013, 2015.
  • Naviworld: localizations for Vietnam and Thailand.
  • Arvato Systems developed a localisation for China

For Nav Partners, who want to develop a localization, they can start with the Microsoft Dynamics Localization Portal – Microsoft Dynamics NAV. They will be assisted by Microsoft (documentations, country specific legal stuff, …).

cheers