README.md 5.51 KB
Newer Older
1
2
# Hardware-In-the-Loop (HIL) network simulation for supporting electrical power grid sub-stations

Johnny Molander's avatar
Johnny Molander committed
3
This is the ns-3 applications from my master thesis conducted in the spring of 2021. The master thesis developed a validated verison of DPDK-enhanced ns-3 capable of supporting multiple NIC. Follow this HowTo to replay the thesis work.
4
5

____
Johnny Molander's avatar
Johnny Molander committed
6
Alterations to the ns-3.33 source code are made in the folder “/ns-3.33/src/fd-net-device/” to:
7
8
9
10
11
12
13
14
15
16

dpdk-net-device.cc/.h

dpdk-net-device-helper.cc/.h

fd-net-device.cc/.h

fd-net-device-helper.cc/.h

____
Johnny Molander's avatar
Johnny Molander committed
17
## HowTo guide for running the thesis simulations
18

Johnny Molander's avatar
Johnny Molander committed
19
### Setting the Ubuntu Environment
20
21
22
23
24

The OS used in the thesis was Ubuntu 20.04.

1. First download this repository to your Ubuntu computer. It contain altered ns-3.33.

Johnny Molander's avatar
Johnny Molander committed
25
26
27
28
    ```sh
    git clone https://gitlab.stud.idi.ntnu.no/johnnymo/masterthesishil.git
    ```

29
2. Install DPDK by using the apt-get command below.
Johnny Molander's avatar
Johnny Molander committed
30
    * DPDK version used in the thesis was 20.11, so to prevent any compatibility issue, I recomend using this version.
31
32
33
    * DPDK can manually be downloaded from [Downloads](https://core.dpdk.org/download/)
    * Check that your NIC is supported by DPDK [Supported HW](https://core.dpdk.org/supported/)

Johnny Molander's avatar
Johnny Molander committed
34
35
36
    ```sh
    sudo apt-get install dpdk
    ```
37
38
39

* Make sure you can run the following from the commandline terminal:

Johnny Molander's avatar
Johnny Molander committed
40
41
42
    ```sh
    sudo dpdk-devbind.py -s
    ```
43
44
45

* If not, add the dpdk commands to path.

Johnny Molander's avatar
Johnny Molander committed
46
3. Alter the GRUB bootloader configuration file to enable hugepages and set driver parameters. This is done by changing the GRUB_CMDLINE_LINUX_DEFAULT parameter in the file “/etc/default/grub”. The provided parameter was my entire parameter setting.
Johnny Molander's avatar
Johnny Molander committed
47
48
49
50
    * ```"maybe-ubiquity"``` was present by default.
    * ```"default_hugepagesz=1G hugepagesz=1G hugepages=1"``` set 1GB hugepage by default and enables 1 hugepage. Tests showed that the application only used 1 hugepage. Set more if needed, but the more you set the more memory will be unavailable by your OS when mounting (see step 5).
    * ```"iommu=pt iommu=on intel_iommu=on"``` are for enabling the vfio-pci (and uio_igb) driver that was used by DPDK in the thesis.
    * ```"isolcpus=4,10,5,11"``` isolate logical cpu cores and prevents the OS from using the logical cores. Add the cores you would like to use in your application here. The thesis used logical cores 4, 5, 10 and 11 that was used to run the DPDK application on.
Johnny Molander's avatar
Johnny Molander committed
51
        * Use ```lscpu``` or ```lstopo``` to see how many logical cpu cores your computer have. 
52

Johnny Molander's avatar
Johnny Molander committed
53
54
55
    ```sh
    GRUB_CMDLINE_LINUX_DEFAULT="maybe-ubiquity default_hugepagesz=1G hugepagesz=1G hugepages=1 iommu=pt iommu=on intel_iommu=on isolcpus=4,10,5,11"
    ```
56
57
58

4. Update GRUB.

Johnny Molander's avatar
Johnny Molander committed
59
60
61
62
    ```sh
    sudo update-grub
    ```
    OR
Johnny Molander's avatar
Johnny Molander committed
63

Johnny Molander's avatar
Johnny Molander committed
64
65
66
    ```sh
    sudo update-grub2
    ```
Johnny Molander's avatar
Johnny Molander committed
67

68
69
70

5. Create a folder you will mount your hugepage(s) on.

Johnny Molander's avatar
Johnny Molander committed
71
72
73
    ```sh
    mkdir /mnt/huge
    ```
74

Johnny Molander's avatar
Johnny Molander committed
75
6. Mount hugepages by default on startup by adding the following line at the bottom of the file “/etc/fstab”.
76

Johnny Molander's avatar
Johnny Molander committed
77
78
79
    ```sh
    nodev /mnt/huge hugetlbfs defaults 0 0
    ```
80
81
82

7. Restart the computer.

Johnny Molander's avatar
Johnny Molander committed
83
### Build ns-3
84

Johnny Molander's avatar
Johnny Molander committed
85
**NOTE:** You must have a c++ compiler (gcc used in thesis) and python version >=3.5.
86
87

1. Go to folder /ns-3.33/.
Johnny Molander's avatar
Johnny Molander committed
88
89
90
    ```sh
    cd ns-3.33
    ```
91
92
93

2. Configure the ns3 *waf* builder.

Johnny Molander's avatar
Johnny Molander committed
94
95
96
    ```sh
    sudo ./waf configure --build-profile=optimized --enable-examples --enable-tests --enable-sudo
    ```
97
98
99

3. Build ns-3 by running:

Johnny Molander's avatar
Johnny Molander committed
100
101
102
    ```sh
    sudo ./waf 
    ```
103
104
105
106
107

* Hopefully ns-3 is able to be built. If not, please refer to [ns-3 Installation guide](https://www.nsnam.org/docs/release/3.33/tutorial/html/getting-started.html)

4. Try to run a sample ns-3 application with:

Johnny Molander's avatar
Johnny Molander committed
108
109
110
    ```sh
    sudo ./waf --run first
    ```
111

Johnny Molander's avatar
Johnny Molander committed
112
### Running DpdkOneNode
113
114
115

**The examples from the thesis are located in /ns-3.33/scratch/**

Johnny Molander's avatar
Johnny Molander committed
116
The simulation used in the thesis was ```dpdkOneNode.cc```
117
118
119

1. To run dpdkOneNode some parameters might need to be changed:

Johnny Molander's avatar
Johnny Molander committed
120
121
* ```dpdkDriver```: Set which driver to use with DPDK.
* ```dev[]```: Array of strings that contain the PCI-address(es) of your NIC(s) to be used in the simulation.
Johnny Molander's avatar
Johnny Molander committed
122

Johnny Molander's avatar
Johnny Molander committed
123
    * You can use the following command in the terminal to get more info about the NIC's and their PCI addresses.
Johnny Molander's avatar
Johnny Molander committed
124

Johnny Molander's avatar
Johnny Molander committed
125
126
127
128
        ```sh
        dpdk-devbind.py -s
        ```

Johnny Molander's avatar
Johnny Molander committed
129
130
131
* ```lCorelist```: Should be the same as you chose to isolate in the GRUB parameter *isolcpus* as it tells the DPDK what cores to use in te simulation
* ```firstLCore```: Sets the logical core of the first NIC you would like to handle incoming traffic on. Must be part of lCorelist.
* ```secLCore```: Sets the logical core of the second NIC you would like to handle incoming traffic on. Must be part of lCorelist.
132

Johnny Molander's avatar
Johnny Molander committed
133
134
135
136
137
2. The *dpdkOneNode* accept input parameter:

* ```pcap```: Boolean if you want to write pcap files from the node in the simulation to file
* ```routeTable``` Boolean if you want to write routing table of the node to file.
* ```runTime```: Sets how long you would like the simulation to run in seconds.
138
139

3. Example of running dpdkOneNode for 30 seconds, writing both pcap and routing tables to file:
140

Johnny Molander's avatar
Johnny Molander committed
141
142
143
    ```sh
    sudo ./waf --run "dpdkOneNode --runTime=30 --pcap --routeTable"
    ```
144

Johnny Molander's avatar
Johnny Molander committed
145
4. For running FdNetDev and TapBridge used in the thesis use the following commands. (Examples with all CLI parameters)
146

Johnny Molander's avatar
Johnny Molander committed
147
148
149
150
151
152
153
154
    ```sh
    sudo ./waf --run "emuFdSwitchTopo --runTime=10 --int0=eth0 --int0=eth1 --pcap --routeTable"
    ```
    OR
    ```sh
    sudo ./waf --run "tapSwitch --pcap --runTime=10" 
    ```
    * The configuration of the environment is explained in the files /ns-3.33/scratch/emuFdOneNode.cc and /ns-3.33/scratch/tapSwitch.cc respectively.