Skip to main content

Migration Workflow

Choose language for code snippet

Python Php Go

In this section we present a workflow example to migrate a third party vendor configuration into a PANOS configuration. This code can be found at the following link:

https://conversionupdates.paloaltonetworks.com/expedition2/examples/workflows.zip

Step 1. Obtaining the API Keys#

Refer to Obtaining the API Keys section to obtain a valid API key stored in the hed variable.

Step 2. Start the Expedition Agent#

Refer to Managing Expedition's Agent section to start the agent and be able to perform imports into a project.

Step 2. Creating an Expedition project#

In the large amount of automation cases, we will require having an Expedition project. Making a POST call to the project route, we can create a project with a desired name. By default, the creator of a project is as well one of the project administrators. Notice that this time we attach the credentials hed in the CURL headers to present our credentials and verify we have permission to create a project.

API syntax for creating a new project:

MethodEndPointParameters
POSThttps://<ExpIP>/api/v1/projectin url
{ "project":"project1", "description":"Project for testing" }
examplehttps://10.0.0.1/api/v1/project{"project":"MyLittleProject", "description":"A migration project"}
print('CREATE NEW PROJECT')data = {"project":projectName}r = requests.post('https://'+ip+'/api/v1/project', data=data, verify=False, headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    #print('New project created successfully')    projectId = json.dumps(response['Contents']['response']['data']['content']['id'])    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))print('')

Step 3. Upload the 3rd party vendor (source) configuration into Expedition#

The migration process would require to upload one of more configuration files to be migrated. A minimum one would be the original vendor configuration file, and we plan to support bringing also routing table files to support dynamic routing tables which values are not specified in the firewall configuration.

The configurations to be migrated must be contained in a Zip file that provides the different configuration files and some instructions on how the migration needs to be performed. More information about this Zip creation can be found in Creating a migration file for a manual Zip creation and in Creating a migration file (by steps) for an assisted Zip creation.

(CASE A) In case of performing the Zip manually, make sure you:

  • name the Zip such as zipArchive_<resourceId>.zip in where resourceId is a numeric value you would later use to refer to the zip resourceId. For instance, zipArchive_1234.zip.
  • locate the zip file within the /home/userSpace/uploads/
  • the zipArchive_<resourceId>.zip has been set to allow www-data to read and write on it.

(CASE B) If you follow the assisted Zip creation, the related API calls will provide you with the Zip resourceId.

Step 4. Converting the source configuration to a PAN-OS configuration#

The migration process takes one compressed ZIP file that encapsulates the file(s) from the original vendor(s) to be migrated, a base configuration where the migration will be inserted and a mapping file that provides instructions to determine where each configuration needs to be placed. For instance, this file will state whether we want a Fortinet policy to become a specific Device Group in a Panorama configuration, if we desire a specific routes file to be used for routing and zones calculations for a specific original configuration or if we desire some remapings to take place for specific networks or interfaces.

Refer to Creating a migration ZIP file to see the list of supported vendors

API syntax for Converting 3rd party vendors' configurations:

MethodURLParameters
POSThttps://<ExpIP>/api/v1/migration/convertin body
{"resource": "Zip resource id",
(optional) dry-run: boolean}
examplehttps://10.0.0.1/api/v1/migration/convertin body
{"resource": 1234,
dry-run: true }
print('CONVERT FORTINET CONFIGURATION TO PALO ALTO')data = {"resource":zipResource}r = requests.post(        'https://'+ip+'/api/v1/migration/convert',         data=data,         verify=False,         headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    #print('Config converted successfully')    print(      json.dumps(         response['Contents']['response']['response-messages']['messages'][0]['message']))print('')

Step 5. Importing the configuration into the Project#

Once the conversion has done, we can import the resulting XML config file into an existing project for later configuration manipulations, such as delete unused objects, rename zones, etc.
The Snippet below shows how to import the converted configuration into a project.

API syntax for Importing the PAN-OS Configuration into the project:

MethodEndPointParameters
POSThttps://<ExpIP>/api/v1/project/{projectId}/importin url
"project": "projectID"

in body
{ "resource": "configuration resource id" }
examplehttps://10.0.0.1/api/v1/project/1/importin url
project=1

in body
{ "resource": "43232" }
print('IMPORT CONFIGURATION')data = {"resource":converterConfig}r = requests.post('https://'+ip+'/api/v1/project/'+projectId+'importConfig', data=data, verify=False, headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    print('Config imported successfully')    #print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))print('')

Assign PAN-OS Device to the Expdition project#

If we want to interact with a device for being able to retrieve it’s configuration, push a new configuration or learn from the traffic logs that the device has generated (this feature will come in future releases), we need to declare a PANOS Device in Expedition.

Step 1. Add a PAN-OS device (Firewall or Panorama)#

The first action is to generate an API call to Expedition to declare the new device. Some initial information regarding the device is necessary, including the serial number, device name, IP address or hostname to be able to reach the device and the device model.

API syntax for adding a PAN-OS device into the project:

MethodEndPointParameters
POSThttps://<ExpIP>/api/v1/devicesin_body
{"serial": "serial#ofyourFW",
"device_name": "a name",
"hostname": "device IP",
"type":"model"}
examplehttps://10.0.0.1/api/v1/devicesin body
{"serial": "01232212",
"device_name": "myFW2",
"hostname": "10.0.0.2",
"type":"pa220"}

The API response will provide us an internal Expedition identifier for the newly generated device.

print('CREATE NEW DEVICE')data = {    "device_name": device_name,    "serial": serial,    "hostname": hostname,    "vsys": vsys,    "type": type}r = requests.post('https://'+ip+'/api/v1/devices', data=data, verify=False, headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    deviceId = json.dumps(response['Contents']['response']['data']['content']['id'])
    #print('New device created successfully')    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message'][0]))print('')  

Step 2. Retrieve Device API Keys#

Once the device is created, if we need to retrieve its content or make future configuration pushes to the device, it will be necessary to have a device API Key.

API syntax for retrieve PAN-OS device API key:

MethodEndPointParameters
POSThttps://<ExpIP>/api/v1/device/keysin body
{"id": "deviceID",
"role": "desired role",
"username": "username in your PANOS device",
"password":"password to access your device"}
POSThttps://10.0.0.1/api/v1/device/keysin body
{"id": "deviceID",
"role": "personal",
"username":"didac",
"password":"kuerti"}

Notice that to be able to retrieve the API keys, we need to provide the login and password that we use to connect with the PANOS device.

When creating and registering the API key, we should provide the role that can make use of it.

There are 4 different options: Admin, User, Viewer, Personal.

info

.admin (project related): all admin users in a given project will use this API key to interact with the device. In the device audit logs, there will be no unique identification of the user that has submitted the API calls to the device, as it could have been done by any of the project administrators.

.user (project related): all users users in a given project will use this API key to interact with the device. In the device audit logs, there will be no unique identification of the user that has submitted the API calls to the device, as it could have been done by any of the project users.

.viewer (project related): all viewer users in a given project will use this API key to interact with the device. In the device audit logs, there will be no unique identification of the user that has submitted the API calls to the device, as it could have been done by any of the project viewers.

.personal (user related): this API key is private to the user that has requested it, and it will be used when this user requires interactions with the Panos Device. A benefit of this key is that in the device audit logs, the user will be identified as having sent the API calls to the device.

The following snippet shows how to consume the Expedition API to retrieve PAN-OS Device API keys and register these API keys into the defined device.

print('ADD DEVICE KEYS')data = {    "id":deviceId,    "role":device_roles,    "auth_type":device_auth,    "username":device_username,    "password":device_pwd}r = requests.post('https://'+ip+'/api/v1/device/keys', data=data, verify=False, headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    #print('Device keys added')    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))print('')

Step 3. Retrieve Device’s content#

We can retrieve the config from a device by specify the value either candidate or running in the {config} parameter when consuming API call.

API syntax for retrieve PAN-OS device contents:

MethodEndPointParameters
GEThttps://<ExpIP>/api/v1/device/{deviceId}/retrieveContent/{config}"deviceID"-> PAN-OS device ID,
"config"-> "running"or "candidate"
examplehttps://10.0.0.1/api/v1/device/1/retrieveContent/runningin url
deviceID=1,
config=candidate

This is a task that, depending on the configuration size and the connection speed with the Panos Device, may take a reasonable amount of time.

Therefore, this task (as several other tasks that require long processing time) is executed as a background task so it does not block the whole script execution.

So, the API call will respond with a job identified jobId that can be checked to identify when the assigned task is completed.

In the second part of the code, we present one approach to monitor the execution state of the download config process, which is informed in the status element within the JSON content response.

print('RETRIEVE DEVICE CONTENT')r = requests.get('https://'+ip+'/api/v1/device/'+deviceId+'/retrieveContent/candidate', verify=False, headers=hed)response=r.json()jobId = json.dumps(response['Contents']['response']['data']['content']['jobId'])print('Job id: '+jobId)#print(json.dumps(response['Contents']['response']['response-messages']['messages']))print('')  
print('CHECK IF DEVICE CONTENT IS DOWNLOADED')r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId, verify=False, headers=hed)response=r.json()jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])percentage = float(jobState) * 100print('Retrieving content from device: '+ str(percentage)+ '%')
#Wait until all content is retrieved from devicewhile(jobState != '1'):    sleep(5)    r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId, verify=False,headers=hed)    response = r.json()    jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])    percentage = float(jobState)*100    print('Retrieving content from device: '+ str(round(percentage,2))+ '%')    print('')  

Step 4. Attach device to the Expedition Project#

While a configuration is being downloaded, for instance, it would be possible to attach the device to an existing project, so we have the chance to import the device configuration into the project or so we can send the resulting project configuration back to the device.

API syntax for attach device to the Expedition project:

MethodEndPointParameters
PUThttps://<YourExpeditionIP>/api/v1/project/in body
{"devices":["deviceID1", "deviceID2"],
"id":"projectID"}
examplehttps://10.0.0.1/api/v1/project/
{"devices": [1],
"id": 1}

In the snippet below, we show how the device created above which id we prior stored in the deviceId variable, is attached to the newly created project. This call is making a modification on the project settings, therefore we are sending a PUT request to the project route.

print('ASSIGN DEVICE TO PROJECT')data = {"devices":[deviceId],"id":projectId}r = requests.put('https://'+ip+'/api/v1/project', data=data, verify=False, headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    #print('Device assigned to project')    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))print('')

Push configuration to a PAN-OS device#

Pushing a project configuration to a device requires having a project created, a device declared and having credentials (Device API keys) to interact with such device.
Once we have those, what would remain is the generation of the different API calls to a PANOS device that will receive the configuration, and executing those calls to apply the changes on the device’s candidate configuration.

Step 1. Generating PAN-OS API calls#

Once we have a project completed and we desire to submit all or part of the configuration to a PANOS device, we will have to generate the API calls. We can create mega, atomic, subatomic and clear API calls.

info

mega: generates one single API call containing the complete XML configuration.

atomic: would generate one API call for each section in the configuration, such as address objects section, service object section, etc. In case of multiple DGs, we may obtain one API call for each section in the DG.

subatomic: would generate an API call for every element in a configuration. For instance, one API call for each address object in a configuration. This may be required when only certain objects should be submitted to the device.

clear: would generate API calls to delete the content of the desired section

API syntax for generating PAN-OS API calls:

MethodEndPointParameters
POSThttps://<YourExpeditionIP>/api/v1/project/{projectId}/apiCalls/{type}in url
"projectId"-> Expedition Project ID,
"type"-> "mega" or "atomic" or "subatomic" or "clear"

in body
{"serial": serial#ofyourFW",
"role": "admin",
"auth_type": "UserPassword",
"username": "YourPAN-OSdeviceLoginusername",
"password": "YourPAN-OSdeviceLoginpassword",
"sourceId": 0,
"action": "set"
}
examplehttps://10.0.0.1/api/v1/project/1/apiCalls/atomic{"serial": 012123212,
"role":"admin",
"auth_type":"UserPassword",
"username":"YourPAN-OSdeviceLoginusername",
"password":"YourPAN-OSdeviceLoginpassword",
"sourceId":0,
"action":"set"
}

The response will provide us a list of the generated API calls with their corresponding ids and types and order of execution.

print('PUSH CONFIG TO DEVICE')data = {"source":source}r = requests.post('https://'+ip+'/api/v1/project/'+projectId+'apiCalls/atomic', data=data, verify=False, headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    #print('Api calls created')    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))print('')

Step 2. Sending API calls to the PAN-OS Device#

Once generated the API calls, we can submit them to a device, either a Firewall or a Panorama. We can provide the device identification via its serial number. In the case a device is managed by a Panorama, this will be used as a proxy to submit the API calls. The Device API key that we have assigned will be used for the submission of the calls. If a personal API key has been registered in Expedition, this will take precedence over an admin/user/viewer API key.

API syntax for sending APIcalls to the PAN-OS Device:

MethodEndPointParameters
POSThttps://<YourExpeditionIP>/api/v1/project/{projectId}/device/pushin url
"projectId"-> Expedition Project ID

in body
{"serial": serial#ofyourFW",
"source": PAN-OSconfigfile
}
examplehttps://10.0.0.1/api/v1/project/1/device/pushin url
"projectId"= 1

in body
{"serial": 01213221,
"source":1
}

When submitting the API calls, we can enumerate those ones that we want to submit or, if none is specified, all the generated API calls will be submitted to the device in the order of execution.

Submitting the API calls is executed as a set of tasks in a job. We can monitor the status of the tasks by checking the job status. Once the process is complete, we can request the result of the execution by checking the complete information of the job.

For instance, in the below snippet, we request information regarding possible errors faced during the submission of the API calls.

data = {"serial":serial,"source":source}r = requests.post('https://'+ip+'/api/v1/project/'+projectID+'/device/push', data=data, verify=False, headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    print('Sending api calls')    print('')jobId = json.dumps(response['Contents']['response']['data']['content']['jobId'])
print('API CALLS STATUS')r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId, verify=False, headers=hed)response=r.json()jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])percentage = float(jobState)*100print('Pushing config to device: '+ str(round(percentage,2))+ '%')
#Wait until all content is retrieved from devicewhile(jobState != '1'):    sleep(5)    r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId, verify=False,headers=hed)    response = r.json()    jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])    percentage = float(jobState)*100    print('Pushing config device: '+ str(round(percentage,2))+ '%')
r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId+'/complete', verify=False, headers=hed)response=r.json()jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])taskCode = json.dumps(response['Contents']['response']['data']['content']['msg']['TasksMsg'][0]['statusCode'])if (int(taskCode)==-1):    taskStatus = json.dumps(response['Contents']['response']['data']['content']['msg']['TasksMsg'][0]['statusMessage'])    taskMessage = json.dumps(response['Contents']['response']['data']['content']['msg']['TasksMsg'][0]['resultCode'])print('Config pushed successfully')

Export PAN-OS configuration from Expedition#

Instead of making API calls from Expedition as stated in previous section, you can export the PAN-OS configuration from Expedition and manually loaded the configuraiton back to PAN-OS device.

The output file path needs to be writable by the www-data user, therefore we recommend to place those files in the project directory.

API syntax for export PAN-OS configuration from Expedition:

MethodEndPointParameters
POSThttps://<ExpIP>/api/v1/project/{projectId}/exportin url
{"projectId"->"$projectId"

in body
{"out:"->PAN-OS configuration file,
"sourceID:"->source configuration ID
}
POSThttps://10.0.0.1/api/v1/project/1/exportin url
"projectId"=1

in body
{"out:"->"toExport.xml",
"sourceID:"1}

print('EXPORT API CALL AS XML') #you can use it to get the final xml and then upload it manually to the firewalloutPath = '/home/userSpace/projecs/fortinetDemo'sourceId = 1 #you can get the source id with the sources given in the previous api calldata = {"out":outPath,"sourceId":sourceId}r = requests.post('https://'+ip+'/api/v1/'+projectId+'/export', data=data, verify=False, headers=hed)response=r.json()success = json.dumps(response['Contents']['success'])if success=='true':    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))print('')