Wednesday, July 19, 2017

#DevOps Series: Upgrading Microsoft Dexterity VSS and TFS repositories to Visual Studio Team Services - Part 2/2

Continuing with our #DevOps series, today we will address the upgrade process from Team Foundation Server (TFS) to Visual Studio Team Services (VSTS). Yesterday, we addressed the upgrade from Visual SourceSafe (VSS) to VSTS - see #DevOps Series: Upgrading Microsoft Dexterity VSS and TFS repositories to Visual Studio Team Services - Part 1/2, - and saw all the important steps needed to ensure your Microsoft Dexterity repository is migrated properly. Likewise, you must observe a series of steps prior to moving your TFS repository to VSTS.



Background

One of the main questions I usually field around this topic is, "Why would I want to move from TFS to VSTS?" The truth is, there are a number of reasons that you may want to consider: a) less server administration, b) a cloud solution gives you immediate access to the latest and greatest features, c) improved developers' connectivity - personally, I love the ability of being anywhere in the world and having access to our repository, without having to establish a VPN connection to some server; and d) if you want to keep the finance people happy, just tell them that you are moving from a CapEx model (servers and hardware that needs to be depreciated, with iffy tax deductions) to an OpEx model (subscriptions that are fully tax deductible).

Once you can see through the benefits, it will be easier to adopt VSTS. The next question is usually, "What are the differences between the two?" For a primer on this, and to keep the article tight, take a look at the following whitepaper:

Fundamental differences between TFS and Team Services


Migrating Microsoft Dexterity repositories from Team Foundation Server to Visual Studio Team Services

As with VSS, there are a few acceptable methods to migrate from TFS to VSTS, as follow:

1) Manually. You can copy the most important and perhaps, the latest projects you are working on. When you are done, you can simply mark off the TFS projects as read only. Under this scenario, the assumption is you will be leaving behind your old TFS server to maintain the history of all your old projects, but if you are getting rid of the server (which is the main reason to move to begin with), you may want to consider a different method.

2) Use the Database Migration tool. As you all know, I am a fan of tools that allow you to automate the process and minimize any risk associated with moving data from one place to another, especially over the internet. The Migration Guide is available here. However, this particular Microsoft tool was very young when we first looked at it, and as of the writing of this article, it was still in preview mode.

3) Use a third-party tool. Frankly, when we did our migration here at Mekorma, we tested a number of tools, but settled on the OpsHub Visual Studio Online Migration Utility, available for free from the Visual Studio Team Services gallery.

Visual Studio Online Migration Utility

OpsHub Visual Studio Online Migration Utility Free Version helps developers migrate the most commonly requested data from an on-premises Team Foundation Server to their Visual Studio Team Services account.  It enables basic migration of history of version control change sets, work items, test cases

The Free Utility is limited to migrating projects with less than 2500 revisions of work items and less than 2500 revisions of source control. The Free utility offers very limited support thru the community supported Q&A forum with no additional support included.

But rather than me trying to describe all the steps, I thought it would be best to embed the demonstration video here:


Tomorrow, I will show you how to leverage Visual Studio Team Services' Build process to extract and chunk your Dexterity applications.

Until next post!

MG.-
Mariano Gomez, MVP

Monday, July 17, 2017

#DevOps Series: Upgrading Microsoft Dexterity VSS and TFS repositories to Visual Studio Team Services - Part 1/2

Yesterday, we talked about #DevOps Series: Microsoft Dexterity source code control with Visual Studio Team Services. The article mainly focused on setting up your Team Services project repository for the first time and taking an existing development dictionary and prepping it and checking in the resources into the repository. But what if you already have a Visual SourceSafe (VSS) or Team Foundation Server (TFS) repository already in place and you are just looking to move to VSTS?



Migrating Microsoft Dexterity repositories from Visual SourceSafe to Visual Studio Team Services

There are two acceptable methods to migrate your Microsoft Dexterity projects repository VSS repository to VSTS: you can use the VSS Upgrade Wizard or you can use the VSSUpgrade command prompt tool. Now, I am a big fan of command-prompt tools, but this is one case where I would suggest you ditch it for the Wizard.

If you would like more information on the VSSUpgrade command-prompt tool, please click here.

Using the VSS Upgrade Wizard 

This is by far, the method I recommend the best. The wizard provides step by step instructions, which makes the process of moving to VSTS a no-brainer. There are a few things you will need to do beforehand.

Preparing for the Upgrade

1.- First, if you are on a version prior to Visual SourceSafe 6.0, you will need to upgrade Visual SourceSafe to version 6, before you can attempt the upgrade. You can download Visual SourceSafe 6.0 here, but please note that this IS NOT an official Microsoft download site, hence, exercise due care when opening any files from an unknown location. Also note that Microsoft support for VSS ended in 2012 - that's right! You are on your own here.

2.- Next, you will need to have a SQL Server available to use as temporary storage to the upgrade process. Since you are already running Microsoft Dynamics GP on some SQL Server, you could probably create a separate instance where you can perform the upgrade. I won't recommend using your production instance to do so.

NOTE: Although SQL Server Express Edition is probably fine for the upgrade, I do recommend you use at the very least SQL Server Standard Edition to prevent any migration issues due to database size limitations imposed by SQL Server Express Edition. If your repositories tend to be very large from years and years of coding (in our case 20 years!) you are probably better off with the Standard Edition of SQL Server.

3.- You will then need to check in all your Microsoft Dexterity project resources into your VSS repository and remove access to all repositories for all developers, but the (main) administrator.

4.- You will have already had to provision a Team Services account. Refer to the previous article in this series for a primer on this process. We found this out the hard way: make sure you create all project shells for your VSS projects before you conduct the upgrade as the Upgrade tool will need this done in advance.

5.- Make a copy of your VSS database and work from the copy. Restore these onto the instance of SQL Server you created in setp 2. Makes sense? Ok, let's move on. As usual, you will not want to expose yourself to some sort of data corruption, so please do not work with your original VSS databases in case something goes wrong. See How To Back Up a Visual SourceSafe Database for additional information on this process.

6.- Download and install the Visual SourceSafe Upgrade Tool for Team Foundation Server (and Visual Studio Team Services). You can get the tool here. You must install the tool on the same machine where you made the copy of your repository database.

7. Run the VSS Analyze Utility to ensure there are no inconsistencies with your VSS database that would prevent the upgrade from being successful. If Analyze produces any errors, you will need to repair the database prior to beginning the upgrade.

7.- For additional preparation steps, please refer to the following MSDN article, Prepare to upgrade from Visual SourceSafe.

Using the Wizard

1. Launch the tool downloaded in Step 6 above. Go to Start and run the VSS Upgrade Wizard.

2. On the Visual SourceSafe Repository page, specify the repository, and if necessary, the Admin password.

Visual SourceSafe Repository page
3. To display the projects in your VSS repository, choose the List Available Projects link. Select the projects you want to upgrade.

List Available Projects
4. Select the check box at the bottom of the page to confirm you have run Analyze. See Step 7 above. Choose Next to proceed.

5. On the Team Project Page, choose Browse and then use the Select a Team project for Migration dialog box to specify the team project into which you want to port the upgraded data. My absolute recommendation here is to select a new team project that you have not been using.

Select a Team Project for Migration page

Choose Next.

6. On the Options page, select whether you want to upgrade the Full history or Tip to omit historical data. When we did this migration, we truncated the data we didn't want to upgrade. That would have been done as an optional step to step 5 above, after all copying the repository database.

Options page
7. On the Options page, specify the name of the SQL Server instance you want the wizard to use for temporary storage.

Options page
Choose Next to continue.

8. Review all settings and choose Next. There will be a checksum to ensure the upgrade can proceed. Choose Upgrade to continue.

9. Once the upgrade is finished, you should be able to navigate to your Visual Studio Team Services account page and verify that all projects have been migrated successfully. If you come across any issues, make sure you print the Migration Report and follow the information provided here to complete additional steps to fix.



Tomorrow, I will walk through the steps to upgrade from TFS to VSTS. Have you completed a migration from VSS to VSTS? I would like to hear your take on it and what "lessons learned" came from executing the migration.

Until next post!

MG.-
Mariano Gomez, MVP

#DevOps Series: Microsoft Dexterity Source Code Control with Visual Studio Team Services

Now that my good friend and fellow MVP, David Musgrave has completed his #MSDexterity Development Environment Series, it's only fitting that I follow it up with a new series on DevOps with Visual Studio Team Services (VSTS) and Microsoft Dexterity.



Background

David's series is important, because it provides an overview of the processes involved in building a Microsoft Dexterity environment and touches upon certain aspects of the code packaging and deployment that are fundamental to DevOps. Keep in mind that no one method is wrong or right, but rather look at it as having a wide array of options at your disposal.

DevOps (Development and Operations) goes beyond the software development life cycle (SDLC) or building a development environment, to incorporate a delivery process that emphasizes communication and collaboration between the product management, software development, and operations teams. Primarily, it seeks to automate the process of software integration, testing, deployment, and infrastructure changes by establishing a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably. We've always heard the cliched term, "People, processes, and technology" thrown around, but DevOps really focuses on bringing the three together.

Since people and processes are highly dependent on each organization, I cannot really cover those here and frankly, they are tons of resources online describing and expanding on the human and process aspects of DevOps. My goal with this series will be to see how we can leverage VSTS as the technology component for a Microsoft Dexterity development operation - mainly, Independent Software Vendors (ISVs), consulting partners, and last, but not least, customers who do have a Microsoft Dynamics GP development team. I suspect some of the principles outlined here will also apply to non-Dexterity development operations, but you need to be the judge as to what and what doesn't apply to your specific case.

I will also assume that most of you reading this will already have a Microsoft Dexterity development environment that relies on either Visual SourceSafe (VSS) or Team Foundation Server (TFS) as your source code control repository and that you are familiar with the concepts of code check-in and check-out.



Implementing Visual Studio Team Services in a Microsoft Dexterity Development Environment

I often get asked, "how do we get started with Visual Studio Team Services?". To put things in perspective, most Dexterity development teams asking this question are always concerned with compatibility between Microsoft Dexterity and Visual Studio Team Services. After all, their current source code control procedures with VSS and TFS have gotten them to this point with little to no issues. The first thing to know is, Microsoft Dexterity communicates to VSTS through the same TFS source control providers defined in the Source Control options. That's right! If you already connect to TFS, you can certainly communicate to VSTS.

Something else to keep in mind is, Microsoft Dexterity versions 12, 14, and 16 do allow you to use the TFS providers directly, this is, without having to install the Dexterity Source Code Control Services (DSCCS) component. Another reminder is, you will need to have the corresponding version of Visual Studio installed on the same machine with Microsoft Dexterity to work with Dexterity TFS provider of your choice. The following table describes the TFS provider and the corresponding version of Visual Studio need to support that particular provider.

TFS Provider
Visual Studio Version
Team Foundation Server 2010Visual Studio 2010
Team Foundation Server 2012Visual Studio 2012
Team Foundation Server 2013/2015Visual Studio 2013 or Visual Studio 2015


Provisioning Visual Studio Team Services

The first thing you will need is to provision Visual Studio Team Services. For this, you will need to go to the Visual Studio website. Once you are on the site, sign in with your organizational account (either Office 365 or Azure AD account).


When clicking on Sign In, you will be challenged for your credentials and will take you to the Get Started page where you can setup your VSTS account. Click on the Create new account button to continue.

Get Stated page
On the Account Creation page, you will setup a project space to host all your Microsoft Dexterity projects and will use this address to administer all your development, QA, and operations users and teams and your Microsoft Dexterity projects as a whole.

Account Creation page

You have a choice to manage your code using Git control or Team Foundation Version Control (TFVC). Git control allows you to setup a distributed source code control model, where each developer has a copy of the source repository on their dev machine; or a TFVC model, where, typically, team members have only one version of each file on their dev machines. Historical data is maintained only on the server. Branches are path-based and created on the server. For more information, take a look at this comparison between Git and TFVC.

After clicking Continue, VSTS will provision your project workspace and your account. Upon completion, a default first project is created for you where you can describe what it's about and setup continues integration for your project (more on that later).

MyFirstProject landing page


Setting up a project using Visual Studio Team Services

The goal of this section will be to show you how to setup your first project using VSTS. The next article in my series will explain how to migrate your VSS and TFS projects to VSTS.

Since no one names their projects "MyFirstProject", it will be necessary to rename this to whatever name you choose. However, for all intends and purposes, let's say this will be our Microsoft Dexterity Quotes Enhancements project. To rename, the project, click on the gear button, and select Overview.

Overview option

You can proceed to change the name of the project under the Profile section, then click the Save button to proceed. Team Services will then present you with a big warning, make sure you read through before you continue. However, since this is our first project and we have nothing else but an empty shell at this point, it is safe to continue with this process.

Project profile page

Rename Project warning

Once the renaming is successful, the profile page will automatically refresh, showing the new project name and the project team name accordingly.


Setting up the Team Services Project Server in the Visual Studio IDE

Since my team normally develops in Dexterity 12 (GP 2013), 14 (GP 2015), and 16 (GP 2016) as our products are supported on three different versions of Dynamics GP, it is necessary for us to have a version of Visual Studio installed that is supported by all versions of Dexterity simultaneously. We have settled on Visual Studio 2012 with Update 5, which allows us to use the Team Foundation Server 2012 provider in all versions of Dexterity mentioned above, although we have side-by-side installations of Visual Studio 2012 and Visual Studio 2015.

Open Visual Studio 2012 and choose Connect to Team Foundation Server on the start page.

Visual Studio 2012 Start page

You will be challenged by Visual Studio to enter your credentials. Enter the credentials you used to provision your Team Services account.


Visual Studio will then take you to Team Explorer once the operation sign in operation has been completed. Proceed to click on Source Control Explorer to view the new project we've setup.

Visual Studio Source Control Explorer
In order to make this fully functional, we need to map our cloud repository to a local path. First, I suggest you setup the local folder using File Explorer. For this case, I will create a folder called MyProjects and under it, a new folder called Quotes. To map the cloud repository, select the root node of the repository, then click the Not Mapped hyperlink above the details pane in Visual Studio. This will open the Map window.

Map window
Point the folder location to the root folder, MyProjects on your local machine. Click the Map button to complete the operation. You will then be asked if you want to download the content of the repo onto your local machine. Since this is a brand new project, there will be nothing in the Quotes project folder (except for the Visual Studio Team Services build templates), so proceed to click Yes to continue. This will now setup the initial project folder.


Setting up Microsoft Dexterity Source Control

When working with integrating Dynamics GP applications, we typically start out from an empty Dynamics.dic development dictionary, but if you already have an existing project dictionary you would like to begin working with, in either cases the following steps will work.

In the case of our example, we will copy the Quotes_dev.dic development dictionary to the MyProjects > Quotes folder. For more information on setting up your development environment, please refer to #Dexterity Development Environments – Part 2: Setting up Projects and follow up to step 10.

Quotes_dev.dic development dictionary

Next, we can launch Dexterity with the dictionary and the Dex.ini file by using our Quotes.exe shortcut. Note that our dictionary already has existing resources we have developed for this project.

Quote_dev.dic inside the Dexterity IDE

To setup the Source Control information, we need to go to Edit > Options in the Dexterity IDE. At first, the provider option will be set to [Disabled]. We can then choose the TFS provider that matches our Visual Studio installation, in this case Team Foundation Server 2012.

Selecting the Source Control Provider in Options window
The root directory, will correspond to the folder with the hierarchy immediately above the project folder, in this case the MyProjects folder created above.

Root directory setup
Since Dexterity uses the TFS providers supplied with Visual Studio, it needs the credentials you use to access the repository. Enter that information in the User Name field. NOTE: the password field remains disabled.

User Name setup
Click on the ellipses button next to Project Name to select the project (do not type). If you can select the project, then it means that Dexterity is communicating with your source control repository - and that's always a good thing :-)

Projects window

Finally, point to the Original Dictionary for which your code was written. As a personal note, I keep a folder on my machine called Dictionaries, which contains sub folders hosting a clean dictionary for each version of Dynamics GP I develop for. This prevent me from accidentally removing, say, the Dynamics GP installation and not having a dictionary available, causing my Dexterity source code control to fail.

Original Dictionary


Synchronizing your dev dictionary with Visual Studio Team Services

To synchronize your development dictionary to the VSTS repository, you must first update your source code control state. This will allow Dexterity to determine which resources are new.

NOTE: New resources are defined by a resource ID of 22,000 and above. Alternate resources, will also appear as new to Dexterity source control.

Update Source Code Control state

Once you update the source code control state, proceed to check in the resources into the repository. Choose Explorer > Source Control > Check In...

Source code check in

If you are going to check in all resources, click the Insert All button. Be sure to enter a comment that describes the best you can what is being done, i.e., "Initial check in of Quotes project resources".

Check in process

Both Visual Studio Source Control Explorer and Team Services should reflect the checked in resources. You can also look to your hard drive, under the Quotes folder to see the new folders created for Base, Forms, Reports, Scripts and Tables resources.

Resources in Repository

My next article will discuss how to migrate from VSS and TFS to VSTS.

Until next post!

MG.-
Mariano Gomez, MVP

Friday, July 14, 2017

Microsoft Dexterity: Customizing Safe Pay file name

Hello all!

It's been a while since I've worked on my blog as there's quite a bit going on these days: numerous projects at Mekorma, preparing to move to my new place, upcoming trips to Colombia and Fargo, ND, etc., so not much time to dedicate to writing.

Anyways, today I wanted to dive into some development. It is very often that I see on forums a request to be able to customize the Safe Pay file name and I wanted to address this with some Dexterity code, but first - you guessed it! - some background.



Background

Safe Pay is part of Microsoft Dynamics GP's Electronic Banking suite of modules, comprised by Electronic Reconcile, EFT Payables and Receivables, and, well, Safe Pay. The goal of Safe Pay is to allow you to create a file that is sent to your bank to confirm the authenticity of checks and EFTs issued to payees.

Trivia: Who was the original author of the Electronic Banking suite?

The Safe Pay Configurator provides an intuitive (in most cases) user interface that allows you to define the specific file format(s) accepted by your bank(s).

Safe Pay Configurator window
Upon configuring the file format, you will then use the Safe Pay Bank Link Maintenance window to group all the checkbooks associated to the same bank into an Upload ID. As part of this setup, you can define a single file name to be generated for this bank. Naturally, most setups will include some company identifier (in a multi-company environment) and the checkbook to at least make the file unique and distinguishable enough.

Safe Pay Bank Link Maintenance window

Finally, the Safe Pay file itself can be generated only after the payment batch has been posted. For this, you must use the Safe Pay - Transaction Upload window. In this window, you will choose the Bank Upload ID defined above, and a cutoff date. Safe Pay will recall the (posted) payment transactions within the last upload date and the cutoff date entered by you. All checks and EFTs available for processing within that date range will show up for processing.

Safe Pay - Transaction Upload window


The Problem and The Solution

Most organizations would like to have at least a date stamp included as part of the file name itself. For the above example, our desired file name would be something like MMM29-FIRSTBANK-YYYYMMDD.txt, where YYYYMMDD represents the current year, month, and date. As you can tell from the Safe Pay Bank Link Maintenance window, it is not possible to achieve this. There isn't even an option to allow you to select other descriptors for the file name. In order to do this, we will use some Dexterity sanScript to add this feature.

NOTE: This article assumes you have a basic understanding of the principles of integrating Microsoft Dynamics GP applications and that you are familiar with the build process (extract and auto-chunk) of a development dictionary. If you need help setting up your development environment, take a look at the #Dexterity Development Environments series on the Winthrop Development Consultants blog by my good friend and fellow MVP, David Musgrave.

In order to change the Safe Pay file name, we will begin by setting a cross-dictionary trigger against the ME_Configurator_Write_Entire_Format global procedure in the Safe Pay dictionary in our Startup script. You can find this out by capturing a script log by using the Microsoft Dynamics GP Script Debugger tool or taking advantage of the GP Power Tools manual or automatic logging capabilities.

global procedure: Startup

{ Script: Startup
  Created by Mariano Gomez, MVP
  This code is licensed under the Creative Commons 
  Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license.
}

local integer lresult;

pragma(disable warning LiteralStringUsed);
lresult = Trigger_RegisterProcedureByName(1235, "ME_Configurator_Write_Entire_Format", TRIGGER_AFTER_ORIGINAL, script Change_Safepay_File_Name);

if (lresult <> SY_NOERR) then
 warning "Trigger registration error for procedure ME_Configurator_Write_Entire_Format in dictionary SafePay (1235): script Change_Safepay_File_Name";
end if; 

pragma(enable warning LiteralStringUsed);

When ME_Configurator_Write_Entire_Format procedure executes, our script processing procedure, Change_Safepay_File_Name, will fire in response, after the original Safe Pay script is executed. This script makes extensive use of standard Microsoft Dynamics GP global functions, which demonstrates how your integrating code can become really compact if you do some upfront work to understand what's available to you in the core Dynamics.dic dictionary file.

global procedure: Change_Safepay_File_Name

{ Script: Change_Safepay_File_Name
Created by Mariano Gomez, MVP This code is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license. }
in string FormatId;
in string SafepayId;
in string UploadId;
in string FilePathName;
out boolean WroteAll;


local boolean  fExists;
local string  sPath, sFileName, sGeneric;
local string  sTimeStamp;
local integer  nStatus;


sGeneric = Path_MakeGeneric(FilePathName);

{ @@@. Verify the generated SafePay file exists and can be open. }
fExists = File_Probe(sGeneric);

if (fExists) then
 sPath = Path_ParsePathFromPath(FilePathName);
 sFileName = Path_ParseFileFromPath(FilePathName);

 { @@@. Get the current time stamp as a string in YYYYMMDD format. We retrieve the time from the SQL Server. 
   We also take advantage of the RW_Pad Report Write function to add the leading zero to single digit months and days
 }
 sTimeStamp = str(year(sysdate(CURRENT_SERVER))) 
   + RW_Pad(str(month(sysdate(CURRENT_SERVER))), LEADING, ZERO_STR, 2) 
   + RW_Pad(str(day(sysdate(CURRENT_SERVER))), LEADING, ZERO_STR, 2);
 
 { @@@. Call the Microsoft Dynamics GP global function FileName_AppendToName to append the string time stamp }
 sFileName = FileName_AppendToName(sFileName, CH_UNDERSCORE + sTimeStamp);
 
 { @@@. Call the Microsoft Dynamics GP global function File_Rename to rename the Safe Pay file with the new name }
 nStatus = File_Rename(FilePathName, sPath + sFileName, true); 
 
end if;

Note that our script processing procedure uses the same parameter list as the ME_Configurator_Write_Entire_Format Safe Pay procedure. This allows us access to the file path and name as configured in Safe Pay.

You can download the extracted dictionary and chunk file here. Feel free to customize the code as you see fit.

Until next post!

MG.-
Mariano Gomez, MVP