Saturday, December 9, 2017

Microsoft Dynamics GP 2018 installation - First Look

Hi everyone! Walk through Microsoft Dynamics GP 2018 installation with me and learn the basic installation steps. Know what to expect as you go through the process, including setting up the account framework, defining account framework segments, deploying SQL Server Reporting Services reports during the installation, setting up the sample company, Fabrikam; login in for the first time and setting up your home page profile.



I will be working on other videos around Microsoft Dynamics GP 2018, so all ideas are welcome in the comment section.

Until next post,

MG.-
Mariano Gomez, MVP

Tuesday, November 28, 2017

Building Microsoft Dexterity Cross-Dictionary Applications with GP Power Tools - Part 2/2

In Part 1 of this two-part series, we saw how to use the Resource Information and the Runtime Execute Setup features in GP Power Tools to gather the name of a List View object and model pass-through sanScript code that would set the image state of a specific line to checked (marked), based on an entity value in the list. As a refresher, the following video shows where we are at this point:


Now this is all good, but technically speaking, this code should be parametrized to allow us to evaluate and mark a number of entities that will be coming from a setup table in my Dexterity application. We would want to iterate through that table, pass in the configured entities as parameters to this code, so they can be evaluated and marked if exists in the list.

In this article, I want to show 2 more GP Power Tools features that will make your cross-dictionary code usable and transferable from GP Power Tools directly into your Dexterity application.


SQL Execute Setup

In the Runtime Execute Setup code we added (shown in the video), we hard-coded the value of the Multi-Entity Management entity to have it marked when it matched a value in the list view. However, hard-coded values of any kind are evil and hardly represent the universe of values that could be selected by a user. Remember also that the primary intent is to transfer our code with as little edits as possible.

First, and to simulate this condition, we would want to build a simple SQL query to return a list of entities that we could use as part of a parametrized list:

1) Click on Options > Scripting > SQL Execute Setup

SQL Execute Setup

It is worth noting at this point that all these options can be accessed from the GP Power Tools navigation bar.

2) In the SQL Execute Setup window, we will want to setup a Script ID, Script Name and the SQL database we want the query to be executed again.

SQL Execute Setup window

3) The list we want is made up of account segment number 3 of our chart of accounts. We will need unique values for this. The following query should get the job done:

SQL Execute Setup window

We need a third column with the key value to be returned. The first two columns will be used as display values for a parametrized list.

Parameter Lists

Parameter Lists are a very interesting concept in GP Power Tools. They allow you to create everything from custom lookups to range values, or discrete values to be passed to anything from Power Tools' Triggers, Runtime Execute  and .NET Execute scripts, and even other SQL Execute scripts. For this particular case, we need to create a parameter list that will display a custom lookup with the SQL Execute script above. This in turn will allow users to interact with our cross-dictionary code, but will also provide absolute validation that our code is working as intended.

1. Click on Options > Scripting > Parameter Lists

Parameter Lists option

2. In the Parameter List Maintenance window, fill out the required fields. Parameter Title and Parameter Instructions will be presented to the user when the custom lookup is activated.

Parameter List setup

3. In setting up the actual parameter, we will need to define a prompt, set the type to Lookup, the mode to Single Field, the Options to Custom Lookup (SQL), Length/Decimal to 5, as shown below.

Parameter configuration

Once you have set the Options to Custom Lookup (SQL), an expansion button becomes enabled to allow you to select a specific SQL Script. Here we will choose the SQL Script created in our previous section.

Parameter List Lookup SQL Script window

4. Click OK to save the selected lookup SQL script and Save to save the new parameter.

5. Back in the Runtime Execute Setup window, we can reload our script then select the newly created parameter list as a Parameter ID to the script.

Parameter ID field selected

6. Now, it's time to insert the parameter instead of the hard-coded string. Highlight the hard-coded string, "20", then click the Parameters button and select the available parameter. The following video shows the full action:


The insert will create a moniker space for the parameter, {%01%}.

7.  The following video shows the script being tested with the parameter.


8. Finally, we will generate the sanScript pass-through code that will be placed in our Dexterity application. This is done from the Script button drop-down list on the toolbar. This final video shows the results of this process.



If you noticed, the cross-dictionary code includes a parameter being passed to the Dexterity execute() function and an inout variable in the pass-through code to use as parameter to the execution. This is definitely a time saver! Now, all is left is to put this code into a function in my dictionary and we are good to go.

I hope you, as a Dexterity developer, consider incorporating GP Power Tools in your arsenal of development tools. Having the ability to model your cross-dictionary code once, test it out, and incorporate it safely and confidently within your dictionary goes a long way and saves a lot of frustrations in the process of developing your solution.

Until next post!

MG.-
Mariano Gomez, MVP

Sunday, November 26, 2017

Building Microsoft Dexterity Cross-Dictionary Applications with GP Power Tools - Part 1/2

As part of an existing project I am currently working on, we need the ability to programmatically select the entities with the Binary Steam's Multi-Entity Management Select Payables Checks alternate window, when running a Decentralize process. In our software, users have the ability to pre-configure and save those settings, and our code needs to be able to set the values within the entity list view and drive the processing by clicking the Build Batch button.

Binary Stream's Select Payables Checks alternate window

Dexterity developers are constantly challenged by the difficulties of creating cross-dictionary code to interact with resource objects in third-party dictionaries, so I thought I would show some really cool features available in GP Power Tools, written by Microsoft MVP, David Musgrave, over at Winthrop Development Consultants, that can help developers test and package their cross-dictionary code more effectively, and without second guessing whether it will work once deployed.

Resource Information window

One of the first challenges when working with third-party dictionaries is to be able to identify the name of the resource we want to target. In this particular case, the resource in question is a list view on the Select Payables Checks alternate window. 

To identify this resource, we will use the Resource Information window in GP Power Tools. Follow these steps:

1. Click on Options > Resource and Security > Resource Information

Resource Information option

2. In the Resource Information window, click the Show currently selected Window and Field Information check mark.

Resource Information window

By selecting this option, Power Tools will now "listen" for any new window that is open and as you click through the fields on the window, it will identify what you are clicking through and display the corresponding field information.
  
3) Open the Select Checks window (Purchasing > Transactions > Select Checks) and navigate to the entity List View. 

Select Payables Checks and Resource Information windows side-by-side

Now, you can see that the List View object I am interested in is called '(L) MainList'.


Runtime Execute Setup

Since we now know the name of the List View object, we can proceed to create some cross-dictionary code that will allow us to navigate the list and set a state image when the entity is value is, tentatively for this example, "20". As a general rule, it is good to add some extra code to capture the index of the state images associated to objects such as list views and tree views.

For this example, the checked image index value is 2, which I have previously researched to be the correct value. Here are the steps to setup the script.

1. Open the Runtime Execute Setup window. Go to Options > Scripting > Runtime Execute Setup.

Runtime Execute Setup

2. For this script, we will setup a Script ID, which we will call MEM_LV and for script name we will set Multi-Entity Management List View. We will want this script to execute in the context of the Multi-Entity Management dictionary, as shown below.

Runtime Execute Setup

Runtime Execute has the ability to setup scripts that can be rolled out to end-users (Published to Executer Window checkmark) or grouped in projects for easy access and export (similar to a Solution is Visual Studio). We can also drive scripts via custom parameters, but I will leave this topic for the second part of this article.

3. Now that we have filled out the required header information, we just need to add some sanScript code that will change the image state to a checkmark for entity "20". That code looks something like this:

Runtime Execute Setup

Note that we are running the change script on the List View object, however, this particular List View does not have any code running behind the change event. It's always good to manually perform a task in the window in question for the object you are interested in, capturing a script log in the process to see if there's any specific code you need to worry about. 

We can now test this code.


But the pass-through code that I need from my Dexterity application needs to parametrize the entity ID since I will be supplying these values from a table to be marked by the code you saw above. Tomorrow I will show you how to leverage both the SQL Execute Setup and the Parameters Lists features in GP Power Tools to take the headache away from this process.

Until next post!

MG.-
Mariano Gomez, MVP

Friday, November 24, 2017

Retrieving dictionary build numbers outside of Dynamics GP - Revisited

Back in June of 2009 - I know, right! Time flies - I wrote an article, Retrieving dictionary build numbers outside of Dynamics GP, in which I described two methods to retrieving the build number of a particular Dexterity dictionary, none of which involved opening Dynamics GP to do so. The first method involved hovering over the dictionary itself, which would display a summary of the properties, including the build number; the second method involved right-clicking on the dictionary, then selecting Properties, then clicking on the Dictionary tab to obtain the Dictionary Version, Name, ID, associated forms and reports dictionary, compatibility ID, and compatibility message.

Sample Dictionary properties tab

Shortly after the publication of this article, I started seeing messages on the forums and discussion boards that the Dictionary tab was not available in certain environments, but I never really paid attention enough to find the reasons why.

Just recently I was checking the Microsoft Dynamics GP Community forum and stumbled upon someone who had ran into this problem in 2010, ensued by some answers asking to check a few registry entries. It wasn't until 2011 that forum user Nathan Hayden provided a complete answer to restore the Dictionary tab in the File Properties window, which I tested and found it to work like a charm.

So here's the solution:

1. Open the Windows Registry (regedit.exe).

2. Under HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.DIC\OpenWithList,  remove all entries, leaving just (Default) and MRUList.


3. Under HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.DIC\OpenWithProgids,  remove all entries, but (Default) and DIC_auto_file.


3. Finally, delete HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.DIC\User Option, if it exists

NOTE: In Windows 10, the key is UserChoice

Disclaimer: Edit the Registry at your own risk. Inappropriate changes to Windows Registry can disable the operating system! Backup your Windows Registry prior to making any changes.

Until next post!

MG.-
Mariano Gomez, MVP

Wednesday, November 22, 2017

Unhandled database exception Save operation on table 'ASI_SOP_HDR_MSTR_Explorer" has caused an unknown error (39)

What if you end up creating a reminder based on a long running SmartList? It had to happen, right? As it turns out, if you create a reminder based on a long running SmartList, you will run into a nasty unhandled exception leading to an error 39, when launching Microsoft Dynamics GP.

In this particular case, the user created a reminder based on the Sales Transactions SmartList to display existing Orders, except they did not filter out just the orders in the work table. When the user attempts to log into Dynamics GP, they get an hourglass cursor, which eventually ends up with the error:

"Unhandled database exception Save operation on table 'ASI_SOP_HDR_MSTR_Explorer" has caused an unknown error (39)."

Since the issue is caused by the reminder calling such a large SmartList query to be executed, it is best to eliminate the offending reminder. You can run the following T-SQL query to identify the reminder:
-- Created by Mariano Gomez, MVP
SELECT * FROM dbo.SY01404 WHERE USERNAME = 'GP_User_ID';

Once you identify the name of the reminder causing the issue, you can run a delete statement to remove it, as follows:

-- Created by Mariano Gomez, MVP
DELETE FROM dbo.SY01404 WHERE Report_Option_Name = 'Reminder_Name' and USERNAME = 'GP_User_ID';

The above queries need to be executed against your system database. I strongly suggest you do this in a test environment first, and backup all your production databases prior to executing the queries there.

Until next post!

MG.-
Mariano Gomez, MVP

Tuesday, November 21, 2017

Copying Quick Links from one Microsoft Dynamics GP user to another

Just recently, I ran across a question on the Microsoft Dynamics GP Community forum asking for a way to copy Quick Links from one user to another.

Users can tailor their Quick Links by clicking on the Customize this page... link on the upper right corner of the Microsoft Dynamics GP homepage.

Quick Links setup

These settings are stored in the syHomePageQuickLinks table (dbo.SY08140) at the system database level.

However, if you want to deploy a set of Quick Links based on a template user ID, it becomes quite the challenge to have to assist each individual user in doing so. The following SQL script allows you to copy all entries from a Source_UserId to a Destination_UserId account in Microsoft Dynamics GP:

-- Created by Mariano Gomez, MVP
USE DYNAMICS
GO

DELETE FROM dbo.SY08140 WHERE USERID = 'Destination_UserId';

INSERT INTO dbo.SY08140 (USERID
      ,SEQNUMBR
      ,TYPEID
      ,CmdID
      ,CmdFormID
      ,CmdDictID
      ,DSPLNAME
      ,ScbTargetStringOne
      ,ScbTargetStringTwo
      ,ScbTargetStringThree
      ,ScbTargetLongOne
      ,ScbTargetLongTwo
      ,ScbTargetLongThree
      ,ScbTargetLongFour
      ,ScbTargetLongFive)
SELECT 'Destination_UserId'
      ,SEQNUMBR
      ,TYPEID
      ,CmdID
      ,CmdFormID
      ,CmdDictID
      ,DSPLNAME
      ,ScbTargetStringOne
      ,ScbTargetStringTwo
      ,ScbTargetStringThree
      ,ScbTargetLongOne
      ,ScbTargetLongTwo
      ,ScbTargetLongThree
      ,ScbTargetLongFour
      ,ScbTargetLongFive
  FROM dbo.SY08140
  WHERE USERID = 'Source_UserId';
GO

If you are needing to transfer just one Quick Link out of many from one user to another, that becomes a bit trickier because you will need to take into account the sequence number at the destination. It would be something like this:

-- Created by Mariano Gomez, MVP
INSERT INTO dbo.SY08140 (USERID
      ,SEQNUMBR
      ,TYPEID
      ,CmdID
      ,CmdFormID
      ,CmdDictID
      ,DSPLNAME
      ,ScbTargetStringOne
      ,ScbTargetStringTwo
      ,ScbTargetStringThree
      ,ScbTargetLongOne
      ,ScbTargetLongTwo
      ,ScbTargetLongThree
      ,ScbTargetLongFour
      ,ScbTargetLongFive)
SELECT 'Destination_UserId'
      ,(SELECT MAX(SEQNUMBR) + 1 FROM SY08140 WHERE USERID = 'Destination_UserId')
      ,TYPEID
      ,CmdID
      ,CmdFormID
      ,CmdDictID
      ,DSPLNAME
      ,ScbTargetStringOne
      ,ScbTargetStringTwo
      ,ScbTargetStringThree
      ,ScbTargetLongOne
      ,ScbTargetLongTwo
      ,ScbTargetLongThree
      ,ScbTargetLongFour
      ,ScbTargetLongFive
  FROM dbo.SY08140
  WHERE USERID = 'Source_UserId' and SEQNUMBR = 8;
GO


In the above example, I am copying just the entry corresponding to Sequence Number 8 from the source user ID to the destination user ID. You also don't want to run a delete for all the destination user entries as this would, well, remove all Quick Links.

Hope you find this script useful.

Until next post!

MG.-
Mariano Gomez, MVP

Tuesday, November 7, 2017

Duplicating a Microsoft Dexterity window on the same form - Part 2/2

In Part 1 of this two-part article, I showed a method using Dexterity Utilities to duplicate a window on the same form. As I also explained, this method is good and, in fact, highly recommended for windows where you have a small amount of fields and window event scripts since the method calls for renaming all of these scripts to match the renamed window.

If you have a more complex window, then perhaps you will want to follow this method:

Method 2 - Manipulating Form Resource File

This method allows you to use the magic of copy/paste and search/replace to quickly duplicate a window within a form. The following steps should get you there:

1. Launch Microsoft Dexterity IDE and open your development dictionary. 

2. Highlight the form resource containing the window you want to duplicate. Highlight the form and click the Export to Text File button the Resource Explorer toolbar.


3. In the Export to Text File window, click OK to perform the export. 


NOTE: The default export path will be where your development dictionary resides.

4. Open the exported form resource file in a text editor and locate the Windows section of the file. Highlight the section corresponding to the window you want to duplicate and copy/paste into a new Notepad window.


5. In the new text editor window, find all occurrences of the window name, and replace with the new name.


Once you've replaced all window name occurrences, scroll to the very bottom of the text editor and locate the Position property. Replace this number with a value of plus 1 the number of windows on the form, i.e., if the form has 1 window, then the new Position for your duplicated window will be 2, if the form has 5 windows, then the new Position will be 6, etc.

6. Copy/paste the replaced content back to the original text editor containing the form resource definition. You will scroll to the bottom of the last Window definition in the file and paste from the clipboard. 


7. Scroll down to the ~EventScripts section, then highlight and copy all EventScripts within that section out to a new window in your text editor. 


8. Paste the copied event scripts into a new text editor window and search the old window name and replace with the window name in all scripts.


9. Copy the content from the new text editor window back into the form resource file. Locate the last EventScript entry within the ~EventScript section and paste there.


10. Finally, save and re-import the form resource file back into your development dictionary. You should now see the duplicated window in the form.

I hope you enjoyed these two methods of duplicating windows within the same form and find it useful in your future projects.

Until next post!

MG.-
Mariano Gomez, MVP

Monday, November 6, 2017

Duplicating a Microsoft Dexterity window on the same form - Part 1/2

Just today, I was trying to solve a problem that at first seemed very simple: I needed to duplicate a Microsoft Dexterity window on the same form.

Background

To provide some context, I have a Dexterity form which hosts a modal window that needs to be displayed in middle of a process - we will call this window, window B of form B. Form B implements an OpenWindow form script that serves as an API to any calling script. The OpenWindow scripts implements a parameter that allows the modal window to either wait for user input or continue processing automatically (that is, without user input), which in turn would close the modal window (and the form) upon completion of all processes executed by the OpenWindow script.

To make matters more interesting, one of the callers happen to be a script on another modal window in another form - we will call this window, window A of form A. Now, when this code (and process) is performed from the Microsoft Dynamics GP desktop client, everything occurs as expected: the field script on window A of form A executes its processes, then calls OpenWindow of form B (in background) and closes. OpenWindow of form B opens window B, executes its processes and closes form B.

When this code is executed under Microsoft Dynamics GP web client, form A with modal window A executes its processes, and although a call to OpenWindow of form B is performed, the modal window on form B never displays because the modal window A of form A never closes.

As it turns out, both the Silverlight (GP 2013/GP 2015) and HTML 5 (GP 2016 and above) web clients have a strict platform implementation for modal windows, this is, they can only be dismissed by user interaction, and not by code - the Dexterity close window button is also disabled for obvious reasons.

So what can a Dexterity developer do if they need to implement a code driven modal window for the web client? Well, you can turn to a mode-less window or a primary window, if needed.


So now the question becomes, how do you duplicate the window on the same form? There are a couple methods you can use to do this:

Method 1 - Dexterity Utilities

1. Create a copy of your development dictionary.



2. Launch the Dexterity IDE. Open the copy of the development dictionary and rename the desired window in the target form. Since I want my window type to be a Modeless Dialog, I change the name to reflect that by adding _ML at the end of the name, and update the actual Window Type property.



NOTE: Once you rename the window, you will want to update all field and window level event scripts to match the new window name.

3. Launch Dexterity Utilities. Open the copy development dictionary as your source and the primary development as your destination.



4. Click the Transfer button on the toolbar and select Window. Locate the renamed window on the desired form and click Transfer.


Enter the name for the transfer report. I strongly suggest you look at the report for any errors.


NOTE: You are *not* presented with a destination form. This is because Dexterity Utilities will transfer the window to a form of the same name in the destination dictionary.

Close Dexterity Utilities.

5. Launch Microsoft Dexterity IDE and open your primary development dictionary. Open the form and now you will see that the Modeless Dialog window is now a part of the form.


You can now compile the form (and the entire dictionary at some point) to ensure all your scripts compile properly and without any errors.

Final thoughts for this method:

1. Since all field event scripts on the window have the same code, consider moving the code in these fields to form level procedures and functions where appropriate. This will ensure any code fixes and expansion is done in one place.

2. I feel this method works well for windows with a very low amount of field scripts. If you have more complex windows to duplicate, then this method becomes a bit more involved, since renaming all field and window level event scripts could be a challenge.

3. This method works to duplicate any window of any window type on the same form.

Tomorrow, I will present a second method that may prove more effective to duplicate windows on the same form, especially for windows with a significant amount of field and window level scripts.

Until next post!

MG.-
Mariano Gomez, MVP

Tuesday, September 26, 2017

Microsoft Dynamics GP Security and Audit Field Manual

My friends, MVP Mark Polino (@mpolino) and Andy Snook (@snookgofast), both members of the Fastpath team, have just released a comprehensive security book titled, Microsoft Dynamics GP Security and Audit Field Manual.

The book can be found in printed and Kindle formats on Amazon.com and I encourage you to get a copy, read up, and put into practice as this book goes beyond the boring task of assigning security to windows and reports just to prevent someone from accessing some area of the application, and into the realms of compliance, separation of duties, and audit controls.


The book can be found in printed and Kindle formats on Amazon.com and I encourage you to get a copy, read up, and put into practice as this book goes beyond the boring task of assigning security to windows and reports just to prevent someone from accessing some area of the application, and into the realms of compliance, separation of duties, and audit controls.

Finally, I want to take the opportunity to thank both Mark and Andy for extending me an invitation to write the foreword to their book. 

Until next post!

MG.-
Mariano Gomez, MVP

#DevOps Series: Building Dexterity Applications with Visual Studio Team Services - Summary





My DevOps series has concluded, although, I believe this will not be the first or the last time I write about this subject. DevOps is here to stay and the tools and technologies to support development teams only keep getting better.

The following is a list of the topics I covered in the series and I encourage you to add your comments to the comment section of the posts that caught your attention. Let me know what you are doing today and how you plan to incorporate DevOps into your development operation processes.

July 17 - #DevOps Series: Microsoft Dexterity Source Code Control with Visual Studio Team Services

July 17 - #DevOps Series: Upgrading Microsoft Dexterity VSS and TFS repositories to Visual Studio Team Services - Part 1/2

July 19 - #DevOps Series: Upgrading Microsoft Dexterity VSS and TFS repositories to Visual Studio Team Services - Part 2/2

Aug 01 - #DevOps Series: Building Dexterity Applications with Visual Studio Team Services Part 1/3

Aug 16 - #DevOps Series: Building Dexterity Applications with Visual Studio Team Services Part 2/3

Sep 25 - #DevOps Series: Building Dexterity Applications with Visual Studio Team Services Part 3/3


I also prepared this video, which I originally intended to add to the previous article, but I am really glad I left it for the summary post. Please be sure to check it out (better viewed in full screen mode).



A link to the Helper.ps1 script containing the library of functions used by the PowerShell scripts found in the previous article can be download from my OneDrive public share, here. These scripts are being updated constantly as we evolve our Build-Engine. Check back for additional updates.

Until next post!

MG.-
Mariano Gomez, MVP

Monday, September 25, 2017

#DevOps Series: Building Dexterity Applications with Visual Studio Team Services Part 3/3

In part 2 of this series, I covered how to setup a Build Definition of for our Build-Engine project. I also began showing the steps required by the Build-Engine definition in order to take your development project from the dictionary to an actual set of extracted dictionaries and chunk files that can be delivered to your QA team.

NOTE: the same process can be used to take your dictionaries from QA to release to your download site.

The first step, as shown before, is to determine the source for the Build-Engine process. We said that we would use the Build-Engine project itself as source for the Build process, since it contains all our Dexterity (and Dexterity Utilities) files, PowerShell scripts, and macros to make it all happen.


1. Following the selection of the source, our first task is to create the necessary folders to host the various files. This tasks uses an inline PowerShell to do this:

CreateFolders (inline PS script)
$folders = @("Build", "Source", "Logs", "Generic", "Temp")  # Create these folders

foreach($item in $folders)
{
    mkdir "$(Get-Location)\$($item)\" -ErrorAction SilentlyContinue | Out-NULL
}

The task creates the following folders:

Build: stores chunk files with no source code

Source: stores the extracted dictionaries and chunk files

Logs: stores all the log files generated by Dexterity Utilities in the process of extracting and chunking dictionaries

Generic: stores the downloaded Dexterity project repository files

Temp: stores any additional component needed throughout the Build process

2. Once we have set up the needed folders, we can then proceed to retrieve our Dexterity project from the VSTS repository. For this purposes, we setup a task that will run our Get_VSTS.ps1 PowerShell script.

Get_VSTS.ps1
Param(
    [string]$SingleModule = "",
    [int]$BuildNumber,
    [string]$VSTSUser,
    [string]$VSTSUserPAToken, 
    [switch]$TestStructure
)

. "$(Get-Location)\Scripts\Helper.ps1"

$modules = Get-ModuleData -Module $SingleModule
if ($modules.Status -ne 0) { 
    Write-Host "Invalid Module : $($SingleModule)" -ForegroundColor Red
    exit 
}

$sourceModule = $modules.SourceFolder
Write-Host "Pulling Module : $($modules.Selected)" -ForegroundColor Green 

# ==============================
# Retrieve source files to pull.
# ==============================
$baseWebFolder = "$/MICR/Base/2/2015B$($BuildNumber)/"
$SourceCodeFolder = "$($baseWebFolder)/$($sourceModule)" # Where to pull from.
$genericFolder = "$(Get-Location)\Generic\" # Where to push files to.

$scopePath_Escaped = [uri]::EscapeDataString($SourceCodeFolder) # Need to have this in 'escaped' form.

Write-Host "`tfrom $($SourceCodeFolder)`n`tinto $($genericFolder)`n"

$recursion = 'Full' # OneLevel or Full
#$recursion = 'OneLevel' # or Full
 
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $VSTSUser, $VSTSUserPAToken)))
 
# Construct the REST URL to obtain the MetaData for the folders / files.
$uri = "https://somedomain.visualstudio.com/DefaultCollection/_apis/tfvc/items?scopePath=$($scopePath_Escaped)&recursionLevel=$($recursion)&api-version=2.2"

# Invoke the REST call and capture the results
$result = $null
$result = Invoke-RestMethod -Uri $uri -Method Get -ContentType "application/json" -Headers @{Authorization=("Basic $($base64AuthInfo)")}

# This call returns the METADATA for the folder and files. No File contents are included.
if ($result.count -eq 0)
{
     throw "Unable to locate code at $($SourceCodeFolder)"
}
$result

# ==============================================
# Create folder structure and sort file objects.
# ==============================================
$script:startTime = Get-Date
$sortedFiles = New-Object 'System.Collections.Generic.SortedDictionary[string, string]'

$_removeLength = $baseWebFolder.Length
for($index=0; $index -lt $result.count; $index++)
{
#    $_path = $result.value[$index].path.substring($_removeLength)
    $_path = "$($genericFolder)$($result.value[$index].path.substring($_removeLength))" -replace "/", "\"
    if ($result.value[$index].isFolder -eq $true) 
    { 
        Write-Host "`t$($_path)" # -BackgroundColor Blue -ForegroundColor Yellow
        New-Item -Force -ItemType directory -Path $_path  | Out-Null
    }
    else
    {
        $sortedFiles[$_path] = $result.value[$index].url
    }
}

# =======================================================
# Create a runspace pool where $maxConcurrentJobs is the 
# maximum number of runspaces allowed to run concurrently    
# =======================================================
$script:maxConcurrentJobs = 10
$script:asyncObj = $null
$Runspace = [runspacefactory]::CreateRunspacePool(1,$script:maxConcurrentJobs)

# Open the runspace pool (very important)
$Runspace.Open()

#$script:Authorization = @{Authorization=("Basic {0}" -f $base64AuthInfo)}
$script:Authorization = @{Authorization=("Basic $($base64AuthInfo)")}
$SortedFiles.GetEnumerator() | foreach {
    # Create a new PowerShell instance and tell it to execute in our runspace pool
    $ps = [powershell]::Create()
    $ps.RunspacePool = $Runspace

    # Base command to 'BeginInvoke'
    # Invoke-RestMethod -Uri $using:remote -Method Get -ContentType "application/json" -Headers @{Authorization=("Basic {0}" -f $using:base64AuthInfo)} -OutFile $using:local 

    [void]$ps.AddCommand("Invoke-RestMethod")
    [void]$ps.AddParameter("OutFile",$_.Key)
    [void]$ps.AddParameter("Uri",$_.value)
    [void]$ps.AddParameter("Method","Get")
    [void]$ps.AddParameter("ContentType", "application/json")
    [void]$ps.AddParameter("Headers", $script:Authorization)

    # Begin execution asynchronously (returns immediately)
    $script:asyncObj = $ps.BeginInvoke() 
}

# ==========================================
## Run the parallel processes to completion.
# ==========================================
if ($script:asyncObj -eq $null) {}
else {
    Write-Host "Pulling $($SortedFiles.Count) code files..."
    while ($script:asyncObj.IsCompleted -eq $false) {}
    Write-Host "`tTime elapsed to pull code: $((Get-Date) - $($script:startTime))"
}

# ================================================
## Change MPP to MMM. Simplifies later processing.
# ================================================
Push-Location $($genericFolder) #Generic 
if (Test-Path "$($genericFolder)\MPP" -PathType Any)
{
    Remove-Item -Path "MMM" -Recurse -ErrorAction SilentlyContinue | Out-Null ## Remove any previous MMM code.
    Rename-Item -path "MPP" -NewName "MMM" 
}
Pop-Location ## Back to where the code was.

This script accepts 5 parameters: the module code (we support 5 products currently) which is validated to prevent an empty parameter from being passed. If we pass in "All", all products will be built; the repository user and personal access token, and a parameter to test the folder structure once it's created. These parameters are passed in by the actual Build definition step.

To retrieve the source files, we construct the service URI and also determine where the files are going to be deposited once retrieved. This is determined by the setting up a relative path to the Generic folder we created in step 1.

Once we connect to the service, we begin retrieving the files by using a for() control structure. There are some other steps that are only relevant to the environment for which this Build process has been designed.

3. Upon retrieving the files from the Dexterity project repository, we are now ready to setup module environment variables and compile the dictionaries, in preparation for the extraction and chunking process.

SetupModuleEnvironment.ps1
Param(
   [int] $VersionNumber,
   [int] $BuildNumber = "000",
   [int] $SubBuildNumber = "0",
   [string] $SingleModule = $null
)

. "$(Get-Location)\Scripts\Helper.ps1"

# Create the folder structures
$folders = Create-FoldersCommands -Version $VersionNumber -Module $SingleModule
Write-Host "Creating Folders:"
foreach ($item in $folders) {
    Write-Host "`t$($item.Folder)"
    mkdir $item.Folder -ErrorAction SilentlyContinue | Out-Null
}


# Copy the files, with replacement of text in text files.
# Ensure the files are saved as 'ASCII'.
$files = Copy-FilesCommands -Version $VersionNumber -Module $SingleModule -BuildNumber $BuildNumber -SubBuildNumber $SubBuildNumber
Write-Host "Copying Files:"
foreach ($item in $files) {
    Write-Host "`tFrom`t$($item.From)"
    Write-Host "`tTo`t`t$($item.To)"
 copy $($item.From) $($item.To)
    Set-ItemProperty $item.To IsReadOnly -value $false

    if ($item.Replacements -ne $null){
        $item.Replacements.psobject.properties | 
        foreach { 
            $_name = "%$($_.name)%"  # The name of the parameter is the text to be replaced, surrounded by '%'
            $_value = "$($_.value)"

            (Get-Content $item.To) -replace $_name,$_value | Set-Content $item.To -Encoding Ascii 
        }
    }
}

Of particular importance is the fact that we use a PowerShell helper script (Helper.ps1) which contains a number of functions that capitalize on the parameters passed here. The general idea, nonetheless, is to make a number of replacements within the macros that assign product information and build numbers, taking into account the version number of Microsoft Dynamics GP for which we will be creating the chunks; and create the shortcuts for Dexterity and Dexterity Utilities to compile and extract the dictionaries, using the proper dictionaries and macros that will run when the Dex platform executables are launched.

In the closing post, summarizing all the articles within this series, I will attach a copy of the Helper.ps1 script.

4. Upon making these replacements and compiling the dictionaries, we can then proceed to extract and chunk our dictionaries.

ChunkDictionaries.ps1
Param(
   [int] $VersionNumber,
   [string] $SingleModule = $null
)
. "$(Get-Location)\Scripts\Helper.ps1"

$EXEx = Create-ExecutableCommands -Version $VersionNumber -Module $SingleModule
Write-Host "Building..."
foreach ($item in $EXEx) {
    Write-Host "$($item.Version)`t$($item.Module)`t$($item.Message)`t" -NoNewline
    Write-Host "`n`t$($item.Executable) : $($item.Timeout) seconds Max.`n`t$($item.Dictionary)`n`t$($item.Macro)`t"

<##>
    # keep track of timeout event
    $timeouted = $null # reset any previously set timeout
    $proc = Start-Process -filePath $item.Executable -ArgumentList @($item.Dictionary, $item.Macro)  -PassThru
    # wait up to x seconds for normal termination
    $proc | Wait-Process -Timeout $item.Timeout -ea 0 -ev timeouted
<##>
    $msg = "Finished."

    if ($timeouted)
    {
        # terminate the process
        $msg = "Time Out!!"
        $proc | kill
    }
    elseif ($proc.ExitCode -ne 0)
    {
        # update internal error counter
        $msg = "Error: $($proc.ExitCode)."
    }
    
    Write-Host "`t$($msg)"
}

Once again, this script takes advantage of the PowerShell helper script library to extract the source code from the development dictionaries and auto-chunk the extracted dictionaries. Note that this script takes in the version of GP to determine the proper version of Dexterity and Dexterity Utilities to launch. This process is completed twice: once for chunks with source (Remove Unused Block in Dexterity Utilities Auto-Chunk option) and another for chunks without source (Total Compression). The source chunks are moved to the Source folder on the Build agent and the object chunks are moved to the Build folder on the Build agent.

NOTE: the Source and Build folders are created by the CreateFolders inline PowerShell script in task 1 above.

5. Upon finalizing the extraction and chunking process of the dictionaries, we move the chunk files with no source code (Total Compression chunks) to the Build sub-folder in the artifacts directory. The artifacts folder is where all resulting files will be stored after the process itself is complete.

Copy Build Artifacts step

6. Then we move the chunks and extracted dictionaries with source code to the Source sub-folder in the artifacts directory.

Copy Source Artifacts
The following Microsoft Docs article talks about Artifacts in Release Management in more detail.

7. Finally, since the Build Agent is volatile, you will need to move the artifacts off the agent and onto a permanent storage location, whether that's on the VSTS servers or a local folder. This is accomplished by publishing the artifacts.

Publish Build Artifacts

My final article in this series will summarize the series and provide links to all previous articles, along with providing a link to the Helper.ps1 PowerShell library.

Until next post!

MG.-
Mariano Gomez, MVP

Wednesday, August 16, 2017

#DevOps Series: Building Dexterity Applications with Visual Studio Team Services Part 2/3

I am so amped-up after my return from the Microsoft Dynamics GP Technical Conference 2017 in Fargo, ND, where I had a chance to catch up with my friends in the partner and ISV community (more on that in a later post). This year, I had a chance to introduce the topics I have been discussing here in my DevOps series and now that I am back, I want to continue writing about the subject as it gets more and more exciting.

In Part 1 of this specific chapter within the series, I talked about building the actual Build-Engine project. If you remember, I specifically said that the build templates provided by Visual Studio Team Services (VSTS) do not fit the bill for Dexterity projects. Dex projects tend to be a bit more cumbersome since we need to have the entire IDE around to compile, extract, and chunk our products. So, it's best if we can isolate these components into an altogether separate project (from that of our actual Dex product) for clarity sake and to maintain our own sanity.


Creating a Build Definition for your Build-Engine Project

Now that we have the Build-Engine project in place, we can proceed to setup a Build Definition. The Build Definition is going to encompass all of the steps required to do things like:

1. Download the resources from our Build-Engine project (Dex IDEs, clean dictionaries, PowerShell scripts, macro templates, etc.

2. Setup any folders needed to support the build process and temporarily store files, etc.

3. Pull the source code from our Dexterity project repository

4. Setup all environment variables

5. Extract dictionaries and create chunk files with (unused blocks) and without source code (total compression).

6. Copy the chunks without source code into an artifact folder

7. Copy chunks and source dictionaries for debugging into an artifact folder

8. Publish the artifact folder

To create a new build for our Build-Engine project:

1. Click on the Build & Release then click the New button.


2. Select an empty template. Dexterity projects, clearly do not conform to any of the existing, pre-defined molds.


3. Click the Apply button to continue.

4. You can now enter the name of your Build-Engine and select from a list of 4 agent queue modes. You can select from Default, Hosted, Hosted Linux Preview, or Hosted VS2017. For all intends and purposes, hosted build agents pools run in the cloud, but can run locally as well. For more information on Hosted Agents, click here. These options define the Build process itself.


The suited option for our Dexterity Build-Engine is Hosted.

5. On the left pane, we can now click on the first task, Get Sources, to identify where the resources for our Build-Engine will come from. In this case, they will come from our Build-Engine project itself, which contains the Dexterity IDEs for versions 12 (GP 2013), 14 (GP 2015), and 16 (GP 2016). All other options are defaulted and really not required to be changed.



This completes the first step (Download Resources for our Build Engine) for today. You can click on Save & Queue to test that all files download properly for the build agent pool.



NOTE: My agent failed in the video as I ran out of allocated build minutes for the month. You will need to assess the length of your build process and ensure you plan accordingly. For more information on Team Services pricing, click here.

I strongly encourage you to read MVP David Musgrave's series on Building a Dexterity Development environment, because all principles used in that series are still applicable in our cloud Build-Engine.

Until next post!

MG.-
Mariano Gomez, MVP