Report Module

Introducing the Report Module

Report module is a powerful engine able to generate, single or multi pages reports in PDF or HTML format, starting from simple DOCX template. It doesn’t require any software installed on the client machine, so it is very well suited for powering web applications, mobile apps and thin clients. Also classic fat client can have a huge benefits from using the Report Module because it really doesn’t add any dependencies on the (already “fat”) client side.

Report Module Main Features

• Generate HTML or PDF reports from DOCX template
• Generate tabular reports
• Generate Master/Detail reports
• Zipped report to reduce payload
• Asynchronous Reports
• Report built in module filters

Required Roles

• report

Predefine Users

DMS provides a predefine user to handle reports. It is on “disabled” state as default:

• user_report: it can create reports. Default passoword “pwd1”;

Report Module Proxy

To generate reports from your client you need to download a proxy. Open you browser and connect to your DMS server typing this url: https://localhost/info.

Your url may be different, change “localhost” with the right DMS Server location.

DMS provides proxies for different languages. Please download “reportrpc” proxy for Delphi language.

GenerateMultipleReport Proxy Method

Let’s give a look at the proxy file generated by DMS. It contains the method GenerateMultipleReport , defined as :

function GenerateMultipleReport(
    const Token: string; 
    const Template: TJsonObject; 
    const ReportData: TJsonObject; 
    const OutputFormat: string): TJDOJsonObject;

Let’s discussing its parameters:

• Token: it’s a string. It’s the session token exchanged between the DMS Server and the client;

• OutputFormat: it’s a string. Its values are: “pdf”, “html”;

• Template: is a JSONObject. This is its structure:

  {
    "report_name":"customerslist",	
    "template_data":"XGSIGIDKENCJALJGLKDJSAREI..."
  }
  

  • report_name: it’s report description name
  • template_data: it is the template file itsef. It is encoded as Base64.

• ReportData: it is a JSONObject. It contains the data that it is used to fill the template with real data. This is its structure:

  { 
    "meta":{},
        "items":[]
  }
  

  • meta: it is a JSONObject. It stores data, you can use in report as a whole. It may contains title, footer descriptor, ect.
  • items: it is a JSONArray. It contains report dynamic data. Each element represents and generates a single report file and contains its data. Data element can be either a JSONObject {} or a JSONArray of objects [{}].

  eg. Let’s suppose you want to generate a report file containing a list of customers

  { 
  	"meta":{},
    	"items":[[{"customer":"Cust1"},{"customer":"Cust2"},...]]
  }
  

  it generates a single report file with all customers.

  Let’s considering this one:

  { 
  	"meta":{},
    	"items":[{"customer":"Cust1"},{"customer":"Cust2"},...]
  }
  

  it generates a report file for each customer.

The method returns a TJSONObject, which may contains either a zipfile or error data. Here an example, extract from samples provided with DMS, that shows how to call this method.

First create the proxy:

// create the proxy
fProxy := TReportsRPCProxy.Create('https://localhost/reportsrpc');
// enable certificates
fProxy.RPCExecutor.SetOnValidateServerCertificate(OnValidateCert);
//trace requests
fProxy.RPCExecutor.SetOnReceiveResponse(
    procedure(ARequest, aResponse: IJSONRPCObject)
    begin
      Log.Debug('REQUEST: ' + sLineBreak + ARequest.ToString(False), 'trace');
    end);

DMS use https protocol. You can use self-signed certificate setting Accepted to True, otherwise you can handle your own certificate using OnValidateCert event.

procedure TMainForm.OnValidateCert(const Sender: TObject; const ARequest: TURLRequest; const Certificate: TCertificate;
var Accepted: Boolean);
begin
  Accepted := true; 
end;

The function below, show how to use GenerateMultipleReport method. You need to provide it:

• a template file (DOCX template file)
• a template format: HTML or PDF
• a JSONObject with the data
• a output file name for the zip file

procedure TMainForm.GenerateReport(const aModelFileName: String; const aFormat: String; aJSONData: TJDOJSONObject;const aOutputFileName: String);
var
  lJTemplateData: TJDOJSONObject;
  lJResp: TJsonObject;
  lArchive: I7zInArchive;
begin
  //Loading template to send to the server
  lJTemplateData := TJDOJSONObject.Create;
  //loding file as Base 64 String
  lJTemplateData.S['template_data'] := FileToBase64String(aModelFileName);
  //send Request to DMS server
  lJResp := lProxy.GenerateMultipleReport('mytoken', lJTemplateData, aJSONData, aFormat);
  try
    //check if theres an error
    if not lJResp.IsNull('error') then
    begin
      raise Exception.Create(lJResp.O['error'].S['message']);
    end;
    //save zip file
    Base64StringToFile(lJResp.S['zipfile'], aOutputFileName);
  finally
    lJResp.Free;
  end;
  // unzip the file
  lArchive := T7zInArchive.Create(I7zInArchive);
  lArchive.classId := CLSID_CFormatZip;
  lArchive.OpenFile(aOutputFileName);
  TDirectory.CreateDirectory(Folder('output_' + aFormat));
  lArchive.ExtractTo(Folder('output_' + aFormat));
end;

function TMainForm.Folder(aFolder: String): String;
begin
  Result := TPath.Combine(TPath.GetDirectoryName(ParamStr(0)), aFolder);
end;

This example uses some DMVCFramework build-in function to encode or decode to Base64, T7zInArchive to zip and unzip and JsonDataObjects to manage JSONObject. Anyway you can use your own class and funcions or delphi predefined library.

Report Templates

Report Module uses Jinja as template language. It has a very light syntax that allows to generate complex reports.

To edit a report template you need to generate DOCX file.

DMS Report Engine uses two main object to inject data inside the report:

• data: it gives access inside the report to the “items” property ;
• meta: it gives access inside the report to the “meta” property.

Here it is some examples.

Single File report with one page

The client sends template data as shown below. “items” property is an array with one element which contains an object with customer info. The element is a single JSONObject that represents Customer info.

{ 
    "meta":{"title":"Customer Detail"},
    "items":
        [
            {
            "cust_no": 1003,
            "customer": "Buttle, Griffith and Co.",
            "contact_first": "James",
            "contact_last": "Buttle",
            "phone_no": "(617) 488-1864",
            "address_line1": "2300 Newbury Street",
            "address_line2": "Suite 101",
            "city": "Boston",
            "state_province": "MA",
            "country": "USA",
            "postal_code": "02115"
            }
        ]
    }

Let’s editing the template just to shows customer’s information on single file report with one page. DMS report engine will inject items object into “data” and “meta” object into “meta”.

										{{meta.title}}

{{data.customer}}

Contact: {{data.contact_first}} {{data.contact_last}}
Phone {{data.phone_no}}
Address: {{data.address_line1}} – {{data.address_line2}} – {{data.city}}
Country: {{data.country}}

The syntax {{data.contact_first}}, allows to print the content of an object property.

This data structure and data generates a single file with a single page with customer information . See picture below.

Single File Report with list of data

The client sends template data as shown below. “items” property is an array with one element which contains an array of customer objects.

{
  "title":"Customer List",  
  "items":  [    
        {
            "cust_no": 1003,
            "customer": "Buttle, Griffith and Co.",
            "contact_first": "James",
            "contact_last": "Buttle",
            "phone_no": "(617) 488-1864",
            "address_line1": "2300 Newbury Street",
            "address_line2": "Suite 101",
            "city": "Boston",
            "state_province": "MA",
            "country": "USA",
            "postal_code": "02115"
        },
      {
            "cust_no": 1004,
            "customer": "Dallas Tecnologies",
            "contact_first": "Glenno",
            "contact_last": "Brown",
            "phone_no": "(214) 960-2233",
            "address_line1": "P.O. Box 47000",
            "address_line2": "",
            "city": "Dallas",
            "state_province": "",
            "country": "USA",
            "postal_code": "02115"
        },
       // ..... 
    ]
}

To render this report we could use:

• List Template
• Table Template

List Template

This type of report is just a simple list. In this example a simple list of customers.

{{meta.title}}

{%for c in data%}
Customer: {{c.customer}}
Contact: {{c.contact_first}} {{c.contact_last}}
Phone {{c.phone_no}}
Address: {{c.address_line1}} – {{c.address_line2}} – {{c.city}}
{% if c.country == “Italy” %}Country: {{c.country}}
{% elif c.country == “USA” %}Country: {{c.country}}
{% else %}Country: {{c.country}}{%endif%}
{%endfor%}

Notice that to iterate through the list it has been used the {%for.. in%} … {%endfor%} statement. There are also some conditional statements {% if … %} {%elif …%} {%endif%}, to address some logical printing.

This data structure and data generates a single file with many pages with customer information . See picture below.

Table Template

This type of report shows data in tabular format: a set of columns and rows. Here it is the template:

Notice the loop is different. This time the statement is {%tr for c in data %}… {%tr endfor%}, but works in the same way, but it is used to repeat the row appearance.

This data structure and data generates a single file with a single or many pages with customer information . See picture below.

Single File for each data

Let’s suppose we want to print each customer record on a single file .The client sends template data as shown below. In this case, “items” property is just a simple customer object array .

{ 
    "items":
        [
          {
            "cust_no": 1003,
            "customer": "Buttle, Griffith and Co.",
            "contact_first": "James",
            "contact_last": "Buttle",
            "phone_no": "(617) 488-1864",
            "address_line1": "2300 Newbury Street",
            "address_line2": "Suite 101",
            "city": "Boston",
            "state_province": "MA",
            "country": "USA",
            "postal_code": "02115"
          },  
          {
            "cust_no": 1004,
            "customer": "Dallas Tecnologies",
            "contact_first": "Glenno",
            "contact_last": "Brown",
            "phone_no": "(214) 960-2233",
            "address_line1": "P.O. Box 47000",
            "address_line2": "",
            "city": "Dallas",
            "state_province": "",
            "country": "USA",
            "postal_code": "02115"
          },
          ....
        ]
    }

Template could be as the following:

{{data.customer}}

Contact: {{data.contact_first}} {{data.contact_last}}
Phone {{data.phone_no}}
Address: {{data.address_line1}} – {{data.address_line2}} – {{data.city}}
Country: {{data.country}}

This data structure and data generates many files with a single page with customer information. A preview of each file may be as the following:

Master/Detail Reports

You may need to print master/detail relationship. Suppose you want to print a report that shows all customer invoices. The “items” property, in this case, should be an array with a single element which contains an array of customer’s invoices objects.

Here it is the data structure:

{
  "items": [
    [
      { 
        "customer": "Signature Design",
        "rows": [
          {
            "product": "Pizza Margherita",
            "price": 10,
            "quantity": 10
          },
          {
            "product": "Pizza Napoli",
            "price": 8,
            "quantity": 12
          },
         // .... ---> other products
        ]
      },
     //....  ---> other customers invoices
        ]
      }
    ]
  ]
}

The template could be similar the following:

Notice here the double loop {%for .. in%} and {%tr for in%}. The first loop is to iterate through customers invoices and the second to print each invoices rows.

Interesting is the use of some specialized function like subtotadd(), subtotget(), subtotclear(), to get the invoice total and the use of filters introduced by pipe ( “|").

This data structure and data generates a single files with many pages with customers invoices . A preview may be as the following:

Asynchronous Reports

So far we have been discussing about synchronous reports. DMS can handle reports generation asynchronously. Asynchronous reports relay on event streams to notify users on report status. Each time a user request for a report, Report Module appends an event on a user queue (queues.jobs.report.<username>) with the current state. Each users has its own report queue.

Reports can be in the following state:

• TO_CREATE: report is waiting to be created by Report Module job
• CREATED: Report has been created
• DELETED: Report has been deleted

To handle asynchronous report you can use these methods:

• GenerateMultipleReportAsync
• GetAsyncReport

GenerateMultipleReportAsync Method

GenerateMultipleReportAsync appends a request to the server to generate the report provided with the data and the output format requested.

Here it is the declaration:

function GenerateMultipleReportAsync(const Token: string;
      const ReportName: string;
      const Template: TJsonObject;
      const ReportData: TJsonObject;
      const UserToNotify: TJsonArray;
      const OutputFormat: string): TJsonObject;				      

Comparing with the synchronous version, it has two new parameters:

• ReportName: it is a string and it the descriptive name of the report to generate
• UserToNotify: it is a JSONArray with the list of users (DMS Users) to be notified

It returns a JSON with the following informations:

{
    "info":"Report equeued",
    "queuename":"queues.jobs.report.user_admin",
    "reportid":"B5BE4E9A97004C18AEDBFF9A2497623F",
   	"reportname": "bank_report_2021_03_04_15",
    "userstonotify": [
		"user_sender",
		"user_admin",
		"user_report"]    
}

Parameters in brief:

• info: it’s a string. It is a message from the server
• queuename: it’s a string and holds the queue name where report state will be appended.
• reportid: it’s a string and it is an internal unique id that DMS gives to the report.
• reportname: it’s a string and it’s the name of the report file
• userstonotify: its a JSONArray with the names of the users that will be able to download the report

GetAsyncReport Method

It downloads an asynchronous report.

  function GetAsyncReport(const Token: string; const ReportID: string): TJDOJsonObject;

It returns a JSONObject with a “zipfile” base64 string property containing a zip file stream with the report in it.

{
    "zipfile":"XJskjdhfksjyudifyhX--"
}

here’s a brief example, to how call the method:

      var lReportFileName := TPath.Combine(FOutputDir, 'output_pdf.zip');
      var lJResp := fProxy.GetAsyncReport(FToken, reportId);
      try
        if lJResp.IsNull('error') then
        begin
          Base64StringToFile(lJResp.S['zipfile'], lReportFileName);
          var lArchive := CreateInArchive(CLSID_CFormatZip);
          lArchive.OpenFile(lReportFileName);
          TDirectory.CreateDirectory(FOutputDir);
          lArchive.ExtractTo(FOutputDir);
          framePDF1.LoadDirectory(FOutputDir);
        end;
      finally
        lJResp.Free;
      end;

After calling the method, we have to save the stream to a zip file and unzip it.

Retrieving Report State

Reports state rely on Event Stream using DequeueMultipleMessage you can read user report queue and get all reports current status. The method returns the following JSONObject:

{
    "data": [
      {
        "messageid": "132593417186220583",
        "createdatutc": "2021-03-04T14:28:38.622+01:00",
        "message": {
          "outformat": "pdf",
          "state": "TO_CREATE",
          "reportid": "D49DA7658045490AAEE1A57768E59C52",
          "queuename": "queues.jobs.reports.user_sender",
          "reportname": "user_admin_2021_03_04_15_23_31_905",
          "creationtime": null
        }
      },
      {
        "messageid": "132593417186220583",
        "createdatutc": "2021-03-04T14:28:38.622+01:00",
        "message": {
          "outformat": "pdf",
          "state": "CREATED",
          "reportid": "D49DA7658045490AAEE1A57768E59C52",
          "queuename": "queues.jobs.reports.user_sender",
          "reportname": "user_admin_2021_03_04_15_23_31_905",
          "creationtime": "2021-03-04T15:23:36.648+01:00"
        }
      },
      {
        "messageid": "132593417186220583",
        "createdatutc": "2021-03-04T14:28:38.622+01:00",
        "message": {
          "outformat": "pdf",
          "state": "DELETED",
          "reportid": "D49DA7658045490AAEE1A57768E59C52",
          "queuename": "queues.jobs.reports.user_sender",
          "reportname": "user_admin_2021_03_04_15_23_31_905",
          "creationtime": "2021-03-04T15:23:36.648+01:00"
        }
      }
    ]
  }

It is an array of messages. Each message holds a single report status. In the example above, we have 3 items for the same reports. It’s not an error. Each time the report goes from a state to another an event message is enqueued.

Here an example:

 lJObjResp := lProxyEventStream.
    DequeueMultipleMessage(FToken, 'queues.jobs.reports.user_sender', lLastMgsID, 1, 5);
  try
    if not lJObjResp.IsNull('error') then
    begin
      Log.Error(lJObjResp.ToJSON(), 'trace');
    end;
    for var i := 0 to lJObjResp.A['data'].Count - 1 do
    begin
      var lState := lJObjResp.A['data'].O[i].O['message'].S['state'];
      if lState = 'TO_CREATE' then
      begin
        //do something
      end
      else if lState = 'CREATED' then
      begin
        //do something
      end
      else if lState = 'DELETED' then
      begin
        //do something
      end;
      lLastMgsID := lObjQueueItem.S['messageid'];
    end;    
  finally
    lJObjResp.Free;
  end;

Deleting Reports

Reports are automatically deleted by DMS Report Module Job, basing on module configuration

Configuring Asynchronous Reports

“job.reports.config.json” configuration file, under “conf” folder, allows you set this configuration properties:

• max_async_report_count: max number of reports to generate for each job process
• reports_ttl_in_minutes: report life time, expressed in minutes. On expiration report will be deleted.

Report Module Built in Filters

Here a list of built in filters that can be used inside template reports:

Full example

Full example code with all the features explained are available in the official samples provided.