Introduction to Network Programmability
As you were able to see in previous sections of this chapter, learning to code and work with programmable infrastructures is very important in today’s environment. You saw the value of using APIs. Whether you have configured large networks in the past or are just getting started, you know that this probably involved a lot of clicking, typing, copying-and-pasting, and many repetitive tasks. Nowadays, modern APIs enable you to complete powerful tasks, reduce all the repetitive work, and save time.
Using APIs, you can make requests like the ones shown in Figure 3-26 in a very simple way.
)
Figure 3-26 Using Network Infrastructure Device APIs
Modern Programming Languages and Tools
Modern programming languages like JavaScript, Python, Go, Swift, and others are more flexible and easier to learn than their predecessors. You might wonder what programming language you should learn first. Python is one of the programming languages recommended to learn first—not only for network programmability, but for many other scenarios.
Combining programming capabilities with developer tools like Git (GitHub or GitLab repositories), package management systems, virtual environments, and integrated development environments (IDEs) allows you to create your own set of powerful tools and workflows.
Another amazing thing is the power of code reuse and online communities. In the past, when you wanted to create some program, you often had to start “from scratch.” For example, if you wanted to just make an HTTPS web request, you had to create code to open a TCP connection over port 443, perform the TLS negotiation, exchange and validate certificates, and format and interpret HTTP requests and responses.
Nowadays, you can just use open-source software in GitHub or simply use packages such as the Python requests package, as shown in Figure 3-27.
)
Figure 3-27 Installing the Python Requests Package Using pip
In Figure 3-27, the Python package called requests is installed using the package manager for Python called pip (https://pypi.org/project/pip). The requests library allows you to make HTTP/HTTPS requests in Python very easily.
Now that you have the requests package installed, you can start making HTTP requests, as shown in Figure 3-28.
)
Figure 3-28 Using the Python Requests Package
In Figure 3-28, the interactive Python shell (interpreter) is used to use (import) the requests package and send an HTTP GET request to the website at https://h4cker.org. The HTTP GET request is successful and the 200 message/response is shown.
Additional information about the Python interpreter can be found at https://docs.python.org/3/tutorial/interpreter.html and https://www.python-course.eu/python3_interactive.php.
When HTTP servers and browsers communicate with each other, they perform interactions based on headers as well as body content. The HTTP Request has the following structure:
The METHOD, which in this example is an HTTP GET. However, the HTTP methods can be the following:
GET: Retrieves information from the server.
HEAD: Basically, this is the same as a GET, but it returns only HTTP headers and no document body.
POST: Sends data to the server (typically using HTML forms, API requests, and the like).
TRACE: Does a message loopback test along the path to the target resource.
PUT: Uploads a representation of the specified URI.
DELETE: Deletes the specified resource.
OPTIONS: Returns the HTTP methods that the server supports.
CONNECT: Converts the request connection to a transparent TCP/IP tunnel.
The URI and the path-to-resource field represent the path portion of the requested URL.
The request version-number field specifies the version of HTTP used by the client.
The user agent is Chrome in this example, and it was used to access the website. In the packet capture, you see the following:
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36\r\n.
Next, you see several other fields like accept, accept-language, accept encoding, and others.
The server, after receiving this request, generates a response.
The server response has a three-digit status code and a brief human-readable explanation of the status code. Then below you see the text data (which is the HTML code coming back from the server and displaying the website contents).
DevNet
DevNet is a platform created by Cisco that has numerous resources for network and application developers. DevNet is an amazing resource that includes many tutorials, free video courses, sandboxes, learning paths, and sample code to interact with many APIs. You can access DevNet at developer.cisco.com.
If you are new to programming and network programmability, you can take advantage of the following DevNet tutorials and learning paths:
Introduction to Coding and APIs: https://developer.cisco.com/startnow
Network Programmability Basics Video Course: https://developer.cisco.com/video/net-prog-basics/
Parsing JSON using Python: https://developer.cisco.com/learning/lab/coding-202-parsing-json/step/1
DevNet GitHub Repositories: https://github.com/CiscoDevNet
DevNet Developer Videos: https://developer.cisco.com/video
DevNet Git Tutorials: https://developer.cisco.com/learning/lab/git-intro/step/1
DevNet ACI Programmability: https://developer.cisco.com/learning/tracks/aci-programmability
Build Applications with Cisco: https://developer.cisco.com/learning/tracks/app-dev
IOS-XE Programmability: https://developer.cisco.com/learning/tracks/iosxe-programmability
Network Programmability for Network Engineers: https://developer.cisco.com/learning/tracks/netprog-eng
Getting Started with APIs
APIs are used everywhere these days. A large number of modern applications use some type of APIs because they make access available to other systems to interact with the application. There are few methods or technologies behind modern APIs:
Simple Object Access Protocol (SOAP): SOAP is a standards-based web services access protocol that was originally developed by Microsoft and has been used by numerous legacy applications for many years. SOAP exclusively uses XML to provide API services. XML-based specifications are governed by XML Schema Definition (XSD) documents. SOAP was originally created to replace older solutions such as the Distributed Component Object Model (DCOM) and Common Object Request Broker Architecture (CORBA). You can find the latest SOAP specifications at https://www.w3.org/TR/soap.
Representational State Transfer (REST): REST is an API standard that is easier to use than SOAP. It uses JSON instead of XML, and it uses standards like Swagger and the OpenAPI Specification (https://www.openapis.org) for ease of documentation and to help with adoption.
GraphQL and queryable APIs: This is another query language for APIs that provides many developer tools. GraphQL is now used for many mobile applications and online dashboards. Many languages support GraphQL. You can learn more about GraphQL at https://graphql.org/code.
APIs often provide a roadmap describing the underlying implementation of an application. API documentation can provide a great level of detail that can be very valuable to security professionals. These types of documentation include the following:
Swagger (OpenAPI): Swagger is a modern framework of API documentation and is now the basis of the OpenAPI Specification (OAS). Additional information about Swagger can be obtained at https://swagger.io. The OAS specification is available at https://github.com/OAI/OpenAPI-Specification.
Web Services Description Language (WSDL) documents: WSDL is an XML-based language that is used to document the functionality of a web service. The WSDL specification can be accessed at https://www.w3.org/TR/wsdl20-primer.
Web Application Description Language (WADL) documents: WADL is also an XML-based language for describing web applications. The WADL specification can be obtained from https://www.w3.org/Submission/wadl.
REST APIs
Let’s take a look at a quick example of a REST API. There is a sample API you can use to perform several tests at https://deckofcardsapi.com. In Figure 3-29, the Linux curl utility is used to retrieve a “new deck of cards” from the Deck of Cards API. The API “shuffles” a deck of cards for you. The deck ID (deck_id) is wkc12q20frlh in this example.
)
Figure 3-29 Using curl to Obtain Information from an API
Suppose that you want to draw a random card from the deck. Since you have the deck ID, you can easily use the command shown in Figure 3-30 to draw a random card.
)
Figure 3-30 Using curl to Obtain Additional Information from the Deck of Cards API
You can see the response (in JSON), including the remaining number of cards and the card that was retrieved (the 9 of spades). Other information, such as the code, suit, value, and images of the card, is also included in the JSON output.
Example 3-1 shows a Python script that you can use to interact with the Deck of Cards API.
Example 3-1 Sample Python Script to Interact with the Deck of Cards API
#!/usr/bin/python
import requests
deck_id = None
def create_deck():
global deck_id
deck_url = "https://deckofcardsapi.com/api/deck/new/"
deck_response = requests.get(deck_url)
deck_data = deck_response.json()
deck_id = deck_data['deck_id']
print("New deck created with ID:", deck_id)
def shuffle_deck():
shuffle_url = f"https://deckofcardsapi.com/api/deck/{deck_id}/shuffle/"
requests.get(shuffle_url)
print("Deck shuffled")
def draw_card():
draw_url = f"https://deckofcardsapi.com/api/deck/{deck_id}/draw/?count=1"
draw_response = requests.get(draw_url)
draw_data = draw_response.json()
if len(draw_data['cards']) > 0:
card = draw_data['cards'][0]
print("The card is a {} of {}".format(card['value'], card['suit']))
else:
print("No more cards in the deck")
def add_jokers():
jokers_url = f"https://deckofcardsapi.com/api/deck/{deck_id}/jokers/?count=2"
requests.get(jokers_url)
print("Jokers added to the deck")
# Display the menu
while True:
print("Omar's Example with the Deck of Cards API. Please select from the following
menu:")
print("1. Create a new deck")
print("2. Shuffle the deck")
print("3. Draw a card")
print("4. Add jokers to the deck")
print("5. Quit")
choice = input("Enter your choice: ")
if choice == "1":
create_deck()
elif choice == "2":
shuffle_deck()
elif choice == "3":
draw_card()
elif choice == "4":
add_jokers()
elif choice == "5":
print("Goodbye!")
break
else:
print("Invalid choice. Please try again.")
The script in Example 3-1 starts by importing the requests module, which is used to send HTTP requests to the Deck of Cards API. If you do not have the requests module installed, you can easily install it with the pip3 install requests command. The script defines a global variable called deck_id, which will store the ID of the deck that the user creates. It also defines four functions: create_deck(), shuffle_deck(), draw_card(), and add_jokers(). Each function corresponds to one of the actions that the user can select from the menu.
The create_deck() function sends an HTTP GET request to the API endpoint https://deckofcardsapi.com/api/deck/new/ to create a new deck of cards. It then extracts the ID of the deck from the JSON response and saves it in the deck_id variable. The shuffle_deck() function sends an HTTP GET request to the API endpoint https://deckofcardsapi.com/api/deck/{deck_id}/shuffle/ to shuffle the deck. The draw_card() function sends an HTTP GET request to the API endpoint https://deckofcardsapi.com/api/deck/{deck_id}/draw/?count=1 to draw a single card from the deck. It extracts information about the card from the JSON response and prints it to the console. The add_jokers() function sends an HTTP GET request to the API endpoint https://deckofcardsapi.com/api/deck/{deck_id}/jokers/?count=2 to add two jokers to the deck.
The script defines a menu that displays the available actions to the user and prompts the user to enter a choice. It uses a while loop to repeatedly display the menu to the user until the user chooses to quit. When the user selects an action from the menu, the script executes the appropriate function based on the user’s choice. After sending the HTTP request, each function extracts information from the JSON response, if necessary, and prints a message to the console indicating that the action was completed. If the user enters an invalid choice, the script prints an error message to the console and displays the menu again.
Using Network Device APIs
Earlier in this chapter you learned that there are several API resources available in many Cisco solutions such as the Cisco DNA Center. The following are a few basic available API resources on the Cisco DNA Center Platform (10.1.1.1 is the IP address of the Cisco DNA Center):
https://10.1.1.1/api/system/v1/auth/token: Used to get and encapsulate user identity and role information as a single value.
https://10.1.1.1/api/v1/network-device: Used to get the list of the first 500 network devices sorted lexicographically based on host name.
https://10.1.1.1/api/v1/interface: Used to get information about every interface on every network device.
https://10.1.1.1/api/v1/host: Used to get the name of a host, the ID of the VLAN that the host uses, the IP address of the host, the MAC address of the host, the IP address of the network device to which the host is connected, and more.
https://10.1.1.1/api/v1/flow-analysis: Used to trace a path between two IP addresses. The function will wait for analysis to complete, and return the results.
There are a dozen (or dozens?) more APIs that you can use and interact with Cisco DNA Center at https://developer.cisco.com/dnacenter. Many other Cisco products include APIs that can be used to integrate third-party applications, obtain information similar to the preceding examples, as well as change the configuration of the device, apply policies, and more. Many of those APIs are also documented in DevNet (developer.cisco.com).
Modern networking devices support programmable capabilities such as NETCONF, RESTCONF, and YANG models. The following sections provide details about these technologies.
YANG Models
YANG is an API contract language used in many networking devices. In other words, you can use YANG to write a specification for what the interface between a client and networking device (server) should be on a particular topic. YANG was originally defined in RFC 6020 (https://tools.ietf.org/html/rfc6020).
A YANG model typically concentrates on the data that a client processes using standardized operations.
Figure 3-31 shows an example of a network management application (client) interacting with a router (server) using YANG as the API contract.
)
Figure 3-31 A Basic YANG Example
A YANG-based server (as shown in Figure 3-31) publishes a set of YANG modules, which taken together form the system’s YANG model. The YANG modules specify what a client can do. The following are a few examples of what a client can do using different YANG models:
Configure: For example, enabling a routing protocol or a particular interface.
Receive notifications: An example of notifications can be repeated login failures, interface failures, and so on.
Monitor status: For example, retrieving information about CPU and memory utilization, packet counters, and so on.
Invoke actions: For instance, resetting packet counters, rebooting the system, and so on.
The YANG language provides flexibility and extensibility capabilities that are not present in other model languages. When you create new YANG modules, you can leverage the data hierarchies defined in other modules. YANG also permits new statements to be defined, allowing the language itself to be expanded in a consistent way.
NETCONF
NETCONF is defined in RFCs 6241 and 6242. NETCONF was created to overcome the challenges in legacy Simple Network Management Protocol (SNMP) implementations.
A NETCONF client typically has the role of a network management application. The NETCONF server is a managed network device (router, switch, and so on). You can also have intermediate systems (often called “controllers”) that control a particular aspect or domain. Controllers can act as a server to its managers and as a client to its networking devices, as shown in Figure 3-32.
)
Figure 3-32 NETCONF Clients, Servers, and Controllers
In Figure 3-32, a node called a “Manager” manages a NETCONF server (router) and two “Controllers,” which are both a server for the Manager and a client for the other network devices (routers).
NETCONF sessions established from a NETCONF client to a NETCONF server consist of a sequence of messages. Both parties send a “hello” message when they initially connect. All message exchanges are initiated by the NETCONF client. The hello message includes which NETCONF protocol version(s) the devices support. The server states which optional capabilities it supports.
NETCONF messages are either a remote procedure call (RPC) or an “rpc-reply.” Each RPC is a request from the client to the server to execute a given operation. The NETCONF rpc-reply is sent by the server when it has completed or failed to complete the request. Some NETCONF rpc-replies are short answers to a simple query, or just an OK that the order was executed. Some are long and may contain the entire device configuration or status. NETCONF rpc-replies to subscriptions consist of a message that technically never ends. Other information of the rpc-reply is generated by the server. A NETCONF rpc-reply may also be a NETCONF rpc-error, indicating that the requested operation failed.
NETCONF messages are encoded in an XML-based structure defined by the NETCONF standard. The NETCONF communication is done over Secure Shell (SSH), but using a default TCP port 830. This can be configured to a different port.
SSH supports a subsystem concept. NETCONF has its own subsystem: netconf. Figure 3-33 shows how you can connect to a networking device (in this case, a CSR-1000v router configured with the hostname ios-xe-mgmt.cisco.com). The username of the router is root. You are also asked to provide a password. The router is configured for NETCONF over TCP port 10000.
)
Figure 3-33 Using the NETCONF SSH Subsystem
An open-source Python library for NETCONF clients called ncclient is available on GitHub at https://github.com/ncclient/ncclient. You can install it using Python pip, as shown here:
pip install ncclient
There are several sample scripts at the DevNet GitHub repositories that can help you get started at https://github.com/CiscoDevNet/python_code_samples_network.
RESTCONF
You already learned that REST is a type of modern API. Many network administrators wanted to have the capabilities of NETCONF over “REST.” This is why a REST-based variant of NETCONF was created. RESTCONF is now supported in many networking devices in the industry.
RESTCONF is defined in RFC 8040 and it follows the REST principles. However, not all REST-based APIs are compatible or even comparable to RESTCONF.
The RESTCONF interface is built around a small number of standardized requests (GET, PUT, POST, PATCH, and DELETE). Several of the REST principles are similar to NETCONF:
The client-server model
The layered system principle
The first two uniform interface principles
One of the differences between RESTCONF and NETCONF is the stateless server principle. NETCONF is based on clients establishing a session to the server (which is not stateless). NETCONF clients frequently connect and then manipulate the candidate datastore with a number of edit-config operations. The NETCONF clients may also send a validation call to NETCONF servers. This is different in RESTCONF.
RESTCONF requires the server to keep some client state. Any request the RESTCONF client sends is acted upon by the server immediately. You cannot send any transactions that span multiple RESTCONF messages. Subsequently, some of the key features of NETCONF (including networkwide transactions) are not possible in RESTCONF.
Let’s take a look at a quick example of using RESTCONF. Example 3-2 shows a Python script that is used to obtain the details of all interfaces in a networking device using RESTCONF.
Example 3-2 Python Script to Retrieve Interface Details from a Networking Device Using RESTCONF
#!/usr/bin/python
import requests
import sys
# disable warnings from SSL/TLS certificates
requests.packages.urllib3.disable_warnings()
# the IP address or hostname of the networking device
HOST = 'ios-xe-mgmt.cisco.com'
# use your user credentials to access the networking device
USER = 'root'
PASS = 'supersecretpassword'
# create a main() method
def main():
"""Main method that retrieves the interface details from a
networking device via RESTCONF."""
# RESTCONF url of the networking device
url="https://{h}:9443/restconf/data/ietf-
interfaces:interfaces".format(h=HOST)
# RESTCONF media types for REST API headers
headers = {'Content-Type': 'application/yang-data+json',
'Accept': 'application/yang-data+json'}
# this statement performs a GET on the specified url
response = requests.get(url, auth=(USER, PASS),
headers=headers, verify=False)
# print the json that is returned
print(response.text)
if __name__ == '__main__':
sys.exit(main())
Figure 3-34 shows the output of the Python script, including the information of all the interfaces in that networking device (ios-xe-mgmt.cisco.com).
)
Figure 3-34 Using Python to Obtain Information from a Network Device Using RESTCONF
OpenConfig and gNMI
The OpenConfig consortium (https://github.com/openconfig) is a collaborative effort to provide vendor-neutral data models (in YANG) for network devices. OpenConfig uses the gRPC Network Management Interface (gNMI). The following GitHub repository includes detailed information about gNMI, as well as sample code (https://github.com/openconfig/gnmi).
The gNMI protocol is similar to NETCONF and RESTCONF. gNMI uses YANG models, but it can be used with other interface description languages (IDLs). The OpenConfig consortium defined several standard YANG models to go with the protocols. These YANG models describe many essential networking features such as interface configuration, routing protocols, QoS, Wi-Fi configurations, and more.