My program is here. The execution process is as follows.
$ python3 Python 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from QuantumQueensE import queens, pprint >>> pprint(queens.sa()) .Q.. ...Q Q... ..Q. >>> exit()
Each square of a 4-by-4 "chess board" is assigned to a row and a column of QUBO (which has 16 rows and 16 columns). The value of each element of the QUBO is 1 when a queen stays there or 0 when no queen stays there. The QUBO consists of three terms. The first term concerns the sum of a row or a column (i.e., Q1 + Q2 + Q3 + Q4). Each sum must be 1, so the term should be (Q1 + Q2 + Q3 + Q4 − 1)2. The second term concerns the sum of squares of diagonal direction (i.e., Q1 + Q2 + Q3 + Q4, Q1 + Q2 + Q3, or Q1 + Q2). Each sum is 0 or 1, so the term should be (Q1 + ... + Qn − 0.5)2. The third term concerns the number of queens, which must be 4. The term should be (Q1 + Q2 + Q3 + Q4 − 4)2. The coefficients of these terms are tentatively 1. I found the third term is not required (i.e., the coefficient may be 0) to solve the four queens.
I have not yet found appropriate values for parameters, such as queens.Ts or the coefficients described above. The program can solve the four queens problem by using default values for the parameters. However, while using the simulater (wildqat), it takes non-short time to get a solution of four queens. If the program is simply extended, it will probably take much time to solve N queens when N is large. This program can probably easily extended to N queens (N > 4), but currently N is fixed to 4. I have not yet tried a D-Wave machine because N is fixed to 4.
P.S. A problem of this program is that the search space is very large because the number of queens (i.e., 4) can be varied, and the number of queens in a row or a column (i.e., 1) is not fixed. The are usually fixed in symbolic programs, so the search is more efficient.
[T-Wave] T-QARD Crews, N-クイーン問題 を D-Waveマシンで解く
[Jha] Rounak Jha, Debaiudh Das, Avinash Dash, Sandhya Jayaraman, Bikash K. Behera, Prasanta K. Panigrahi, A Novel Quantum N-Queens Solver Algorithm and its Simulation and Application to Satellite Communication Using IBM Quantum Experience
[Torggler] Valentin Torggler, Philipp Aumann, Helmut Ritsch, and Wolfgang Lechner, A Quantum N-Queens Solver, Quantum Journal, accepted.
]]>A file of Repeater firmware for linear version is available at Github, but an updated version of this firmware for pulley version did not work well. So, now I use an original Repetier 0.93 based version, which the patches were applied.
I could hardly adjust the height of the nozzle when x and y coordinates are not zero by using the program downloaded from Github by unknown reasons, but I could do it by using Repeater 0.93 based version. It was not easy to fix the problem that inputting home (M28) commands repeatedly causes hanging but finally found a set of parameters that fix the problem.
Most of updated configurations are included in Configure.h, but temperature error detection is disabled in Extrude.cpp. (Few temperature errors occur while using PLA filament, but, if a material such as polycarbonate that requires high temperature is used, temperature errors easily occur.) In the original program, if the temperature does not increase ten times while heated, the heater is switched off. However, because strong cooling causes such a condition, this switching-off mechanism is disabled in the updated firmware.
]]>clip(inFile, outFile, options...)
inFile is the input file, andoutFile is the output file. outHeightMin and outHeightMax specifies the vertical range of the output frame. outWidth is the width of the output.
]]> ### Clipping video -- reducing the vertical size ### import cv2, cv, time import numpy as np def createPedestrianWriter(outFile, width, height): writer = cv2.VideoWriter() writer.open(outFile, cv.CV_FOURCC('P','I','M','1'), fps = 30, frameSize = (width, height), isColor = False) return writer def clip(inFile, outFile, outHeightMin = 80, outHeightMax = 280, outWidth = 1280): reader = cv2.VideoCapture(inFile) writer = createPedestrianWriter( outFile, width = outWidth, height = outHeightMax - outHeightMin ) ret, oframe = reader.read() while ret: frame = cv2.cvtColor(oframe, cv2.COLOR_BGR2GRAY) writer.write(frame[outHeightMin:outHeightMax]) ret, oframe = reader.read() ]]>import caltechVideo caltechVideo.genVideo('set00', 'V000')
### Generate an AVI file from Caltech Dataset ### import cv2, cv, time, struct import Image, utils, random, os, gzip import numpy as np def createPedestrianWriter(outFile, width, height): writer = cv2.VideoWriter() writer.open(outFile, cv.CV_FOURCC('P','I','M','1'), fps = 30, frameSize = (width, height), isColor = False) return writer def genVideo1(inFile, outFile): def read_header(ifile): feed = ifile.read(4) norpix = ifile.read(24) version = struct.unpack('@i', ifile.read(4)) length = struct.unpack('@i', ifile.read(4)) assert(length != 1024) descr = ifile.read(512) params = [struct.unpack('@i', ifile.read(4))[0] for i in range(0,9)] fps = struct.unpack('@d', ifile.read(8)) # skipping the rest ifile.read(432) image_ext = {100:'raw', 102:'jpg', 201:'jpg', 1:'png', 2:'png'} return {'w':params[0], 'h':params[1], 'bdepth':params[2], 'ext':image_ext[params[5]], 'format':params[5], 'size':params[4], 'true_size':params[8], 'num_frames':params[6]} print('From {} to {}'.format(inFile, outFile)) params = read_header(open(inFile, 'rb')) bytes = open(inFile, 'rb').read() writer = createPedestrianWriter(outFile, 640, 480) # this is freaking magic, but it works extra = 8 s = 1024 seek = [0]*(params['num_frames']+1) seek[0] = 1024 # print("Params:{0}".format(params)) #!!!!!! for i in range(0, params['num_frames']-1): ##!!! why -1? tmp = struct.unpack_from('@I', bytes[s:s+4])[0] s = seek[i] + tmp + extra if i == 0: val = struct.unpack_from('@B', bytes[s:s+1])[0] if val != 0: s -= 4 else: extra += 8 s += 8 seek[i+1] = s nbytes = struct.unpack_from('@i', bytes[s:s+4])[0] # print("Frame:{0} seek:{1}".format(i, s)) I = bytes[s+4:s+nbytes] tmp_file = '/tmp/img{0}_{1}.jpg'.format(os.getpid(), i) open(tmp_file, 'wb+').write(I) img = cv2.cvtColor(cv2.imread(tmp_file), cv2.COLOR_BGR2GRAY) writer.write(img) return def genVideo(dir, name): inFile = dir + '/' + name + '.seq' outFile = 'Images/' + dir + '_' + name + '.avi' genVideo1(inFile, outFile)]]>
When using images for learning data, GPU memory is much more consumed when pixels are represented by 32-bit floating numbers because each number consumes 4 bytes. To reduce memory consumption, 16-bit floating number is supported by systems such as NVIDIA cuDNN ver. 2 as a predefined data type. However, this method still consumes multiple bytes of memory for each number, and systems such as Theano do not support 16-bit float number. If a pixel can be represented by an 8-bit integer when it is 256 shades. So, if it can be packed to GPU memory, much more data can be stored.
Several Theano programs for deep learning are described in Deep learning tutorial. This tutorial contains a program and explanation of logistic regression. This program contains a function called load_data(), which reads a data set into the GPU memory. This function generates a 32-bit floating array. However, by slightly modifying this function, an 8-bit integer array can be generated instead. The program (function) that generates a (32-bit floating) array, which is included in load_data(), is as follows.
def shared_dataset(data_xy, borrow=True): data_x, data_y = data_xy shared_x = theano.shared(numpy.asarray(data_x, dtype=theano.config.floatX), borrow=borrow) shared_y = theano.shared(numpy.asarray(data_y, dtype=theano.config.floatX), borrow=borrow) return shared_x, T.cast(shared_y, 'int32') # “T” means Theano
This can be modified as follows, then shared_x becomes an 8-bit integer array. It is converted to a 32-bit floating array just before using it. (Otherwise, if trying to use without conversion, Theano outputs error messages.) The values of the array must be, of course, normalized into range −128 to 127. (For example, if the original value is uint8, 128 must be subtracted.)
def shared_dataset(data_xy, borrow=True): data_x, data_y = data_xy shared_x = theano.shared(numpy.asarray(data_x, dtype='int8'), borrow=borrow) shared_y = theano.shared(numpy.asarray(data_y, dtype='int32'), borrow=borrow) return T.cast(shared_x, 'float32'), shared_y
If shared_x is replaced by a mathematical expression as above, get_value function cannot be used for it. So a modification shown below is required.
train_set_x, train_set_y = datasets[0] # datasets[0] is a return value of load_data() (i.e., shared_x) ... # compute number of minibatches for training, validation and testing n_train_batches = train_set_x.get_value(borrow=True).shape[0] / batch_size ...
This program becomes as follows.
train_set_x, train_set_y = datasets[0] ... # compute number of minibatches for training, validation and testing n_train_batches = train_set_y.eval().shape[0] / batch_size ...
In this program, not only get_value(borrow=True) is replaced by eval(), but also train_set_x is replaced by train_set_y. The purpose of this replacement is to reduce the overhead.
Floating numbers can be normalized so that the mean value becomes 0; however, it is difficult for integers, so it would be better to subtract the mean value just before using it. In addition, it would be better to adjust the range to be between −1 and 1. The input variable is named x in this tutorial, so x is to be replace by ((x − T.mean(x)) / 128.0). For example, in the program of logistic regression, the line below is to be replaced.
classifier = LogisticRegression(input=x, n_in=28 * 28, n_out=10)
This must be replaced by the following line.
classifier = LogisticRegression(input=((x − T.mean(x)) / 128.0), n_in=28 * 28, n_out=10)
In the modified program, certain amount of computation is required every time contents of an array is extracted, the ratio of the overhead is less than 10% when the computation is large scale such as that of a convolutional neural network.
]]>Caltech video is in so-called "seq" format. A program that converts it to a format readable by Python programs is available at the following URL. reading .seq files from caltech pedestrian dataset
I used this program and found that it cannot read the last frame of each file correctly, so an error occurs. However, other frames can be read correctly, so I imitate this program for handling the files.
Caltech dataset includes a file of annotations. This file contains bounding box information; that is, rectangles that enclose pedestrians. They are in so-called "vbb" format, which is a binary Matlab format. A binary format is difficult to be handled, so I converted the files into text format. The program called code3 in Matlab (two functions in file named "vbb.m" in the directory), which is linked from the Caltech dataset page, can be used for converting them. The two functions are one that reads binary vbb file (A = vbbLoad(file)) and one that writes text-format vbb file (vbbSaveTxt(A, textFileName, timeStamp)).
The files converted to text format can be handled (further converted) by my programs. Because pattern matching in Python is complicated, I used Perl for converting them to Python format. The following Perl program generates a Python program.
### Bounding box extractor for textual VBB file ### # # Public domain program # coded by Yasusi Kanada # 2015-6-22 open(input, "annotations/${ARGV[0]}/${ARGV[1]}vbb.txt"); print "${ARGV[0]}_${ARGV[1]}=[\n"; while (<input>) { if (/^lbl='(person(-fa|\?)?|people)'\s+str=(\d+)\s+end=(\d+)\s+hide=(\d+)/) { $type = $1; $str = $3; $end = $4; $hide = $5; $pos = ''; $posv = ''; $occl = ''; $lock = ''; } if (/^(pos|posv)\s*=\s*\[(([-\d\w\.\;\s])*)\]/) { $name = $1; $text = $2; $text =~ s/;\s+/\] \[/g; $text =~ s/\s+/, /g; $text = "[[${text}]]"; $text =~ s/, \[\]//; if ($name eq 'pos') { $pos = $text; } else { $posv = $text; } } elsif (/^(occl|lock)\s*=\s*\[(([-\d\w\.\s])*)\]/) { $name = $1; $text = $2; $text =~ s/\s+/, /g; $text = "[${text}]"; if ($name eq 'occl') { $occl = $text; } else { $lock = $text; print " \{'type':'$type', 'firstFrame':$str, 'lastFrame':$end, 'hide':$hide,\n"; print " 'pos':$pos,\n 'posv':$posv,\n"; print " 'occluded':$occl,\n 'lock':$lock\},\n"; } } } print "]\n";
The generated program contains a list per pedestrian. The original file contains separated pedestrian (person) data, and also contains data of people and "person-fa". For each pedestrian, this file contains the first frame number, the last frame number, and the bounding box information of the frames in between: the location (x and y) and the size (width and height). However, this program still have problem (bug); it cannot read some of the files.
]]>The usage of the program is, for example, as follows. (The version of Python can be 2 or 3.)
python naturalDesign.py PLA 0.15 0.05
For the above command, a PLA filament is to be used and the print pitch (layer pitch), 0.5 mm, and the average cross section of the filament, 0.05 mm2, are specified in the program. ABS can be used instead of PLA. More detailed parameters can be specified as follows.
python naturalDesign.py ABS 0.2 0.05 180 4800 120
Here, except the print pitch 0.2 and the cross section 0.05, the number of divisions of a circle is specified as 180 (i.e., it is approximated by 180 lines), head motion speed 4800 mm/min, number of repetitions in the spiral 120 are specified. These are default values, so they can be omitted. (The parameters in between specified parameters cannot be omitted.)
Note that I have not written any patents or similar intellectual properties on this issue because fluctuated 3D printing is caused by nature.
]]>Note that part of the design and printing method that is used in this library and that uses this library is patented by the author.
]]>### Ethernet packet sender/receiver (for Linux raw socket) ### # # Public domain software # # Coded by Yasusi Kanada # 2014-7-29 import optparse, socket, time, binascii BUF_SIZE = 1600 # > 1500 ETH_P_ALL = 3 # To receive all Ethernet protocols # Interface = "eth0" Interface = "eth1" host = socket.gethostbyname(socket.gethostname()) ### Packet field access ### def SMAC(packet): return binascii.hexlify(packet[6:12]).decode() def DMAC(packet): return binascii.hexlify(packet[0:6]).decode() def EtherType(packet): return binascii.hexlify(packet[12:14]).decode() def Payload(packet): return binascii.hexlify(packet[14:]).decode() ### Packet handler ### def printPacket(packet, now, message): # print(message, len(packet), "bytes time:", now, # "\n SMAC:", SMAC(packet), " DMAC:", DMAC(packet), # " Type:", EtherType(packet), "\n Payload:", Payload(packet)) # !! Python 3 !! print message, len(packet), "bytes time:", now, \ "\n SMAC:", SMAC(packet), " DMAC:", DMAC(packet), " Type:", \ EtherType(packet), "\n Payload:", Payload(packet) # !! Python 2 !! def terminal(): # Parse command line parser = optparse.OptionParser() parser.add_option("--p", "--port", dest = "port", type="int", help = "Local network port id") parser.add_option("--lm", "--lmac", "--localMAC", dest = "lmac", type="str", help = "Local MAC address") parser.add_option("--rm", "--rmac", "--remoteMAC", dest = "rmac", type="str", help = "Remote MAC address") parser.add_option("--receiveOnly", "--receiveonly", dest = "receiveOnly", action = "store_true") # parser.add_option("--promiscuous", dest = "promiscuous", action = "store_true") parser.set_defaults(lmac = "ffffffffffff", rmac = "ffffffffffff") opts, args = parser.parse_args() # Open socket sock = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(ETH_P_ALL)) sock.bind((Interface, ETH_P_ALL)) sock.setblocking(0) # Contents of packet to send (constant) sendPacket = binascii.unhexlify(opts.rmac) + binascii.unhexlify(opts.lmac) + \ b'\x88\xb5' + b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f' # Repeat sending and receiving packets interval = 1 lastTime = time.time() while True: now = time.time() try: packet = sock.recv(BUF_SIZE) except socket.error: pass else: dmac = DMAC(packet) printPacket(packet, now, "Received:") if not opts.receiveOnly: if now > lastTime + interval: sendBytes = sock.send(sendPacket) printPacket(sendPacket, now, "Sent: ") lastTime = now else: time.sleep(0.001001) else: time.sleep(0.001001) terminal()
Packets to be sent contains 0x88b5 as its Ethernet type, and 0x000102… as its contents. You can easily change them.
The above code is for Python 2, but it can be used for Python 3 by modifying only "printPacket". The code for Python 2 is to be commented out and the code for Python 3 is to be used. When communicating by Ethernet, the interface name must be specified. This program specifies "eth1", but you can easily rewrite the value of variable "Interface".
The usage is as follows. The file name of the above program is assumed to be "term.py". Simple unicast communication (sending and receiving) can be done as follows.
python eterm.py --lm 00123456789a --rm 00a987654321
Here, the MAC address must be specified. (it can be obtained and copied using ifconfig command).
If the remote address is omitted as follows, the program broadcasts packets. (The local address can be omitted too).
python eterm.py --lm 00123456789a
To receive packets without sending ones, the usage is as follows.
python eterm.py --receiveOnly
The above program can be executed by two machines to send and to receive packets each other. Otherwise, if one machine only receives packets and the other sends and receives packets, a one-way communication can be tested. Such one-way communication is useful for testing Ethernet switch functions.
]]>There are simulators that exactly simulates Ethernet networks, but I could not find an appropriate simulator for educational purposes, so I wrote one by myself. You can open multiple command windows and each window can run this program to simulate a switch or a terminal. It is a very simple program, so there are restrictions such as the port number range. However, you can roughly see mechanisms of Ethernet network.
]]>The usage is written in the read me file in the zip file of the simulator. A simple use case (in Japanese) is available here. The simulator is written by Python (3), so it can run on either Windows, Linux, or Macintosh. It can run on Python 2, but you have to rewrite the program according to the comments in the program.
A paper on this simulator is available, and the history of this program is written in "the lecture blog (in Japanese).
]]># addgreip REMOTE_IP LOCAL_IP GRE_KEY LOCAL_SLICE_IP ip tunnel add gre0 mode gre remote ${1} local ${2} key ${3} ip addr add ${4} dev gre0 ip link set gre0 up ip link set gre0 mtu 1460
As described in the first line, the command arguments are the destination IP address (i.e., the end of the GRE tunnel), the source (my) IP address (i.e., the beginning of the GRE tunnel), the GRE key, and the source (my) IP address on the tunnel (i.e., the IP address of the virtual network) and the subnet length. If the subnet length is not specified, an error occurs.
If no GRE key is specified by the ip tunnel command, the packet size becomes smaller; it must be specified for the above script. An example of the command usage follows.
./addgreip 10.0.32.129 10.0.32.15 1 192.168.10.2/24
The next script is for IP/Ethernet/GRE/IP.
# addgreether REMOTE_IP LOCAL_IP GRE_KEY LOCAL_SLICE_IP ip link add gre1 type gretap remote ${1} local ${2} key ${3} ip addr add ${4} dev gre1 ip link set gre1 up ip link set gre1 mtu 1450
The command arguments are similar to those for the IP/GRE/IP commands. The MAC address specified in the inserted MAC header is filled by the computer if is is not specified by the user. An example of the command usage follows.
./addgreether 10.0.32.129 10.0.32.15 1 192.168.10.2/24
The ip command may refuse the above commnad (type gretap) when using older Linux. In such a case, a package called "iproute2" should be upgraded.
When IP/GRE/IP is used, an ip tunnel command is used for configuration. However, when IP/Ethernet/GRE/IP is used, an ip link command is used. As for command keywords, a "mode" parameter is used for GRE when IP/GRE is used. However, "type gretap" is used when IP/Ethernet/GRE is used. I am skeptical why these commands are so different.
]]>./ethernode <FirstInterface> <LastInterface>
or
./ethernode <FirstInterface> <LastInterface> <PrintLevel>
Because the PC is used as a switch, it must have at least three network interfaces. If the specified range is 1..3, the names of the interfaces are eth1, eth2, and eth3. It is a good idea to reserve eth0 for control purpose. PrintLevel is used for controlling messaging on packet transmission, learning, and packet contents. If 0 is specified, nothing is outputted. If 1 to 4 is specified, messages are outputted. (See the program for detail.)
/*** * * Software-based Ethernet Switch * * Coded by Yasusi Kanada * Ver 0.1 2011-1-11 Initial version * Ver 1.0 2011-1-16 48-bit ID splitted * Ver 1.1 2011-2-23 Learning/traffic statistics function introduced * Ver 1.11 2011-2-24 Debugged and main-loop optimized * Ver 1.12 2012-5-20 Cleaned * ***/ #include "Ether.h" #include <fcntl.h> #include <sys/time.h> #define bool int32_t #define true 1 #define false 0 // #define DEBUG 0 #define MAX_PACKET_SIZE 2048 // Sufficiently larger than the MTU // Timeout time in usec #define LearningTimeout 15000000 #define ReferenceTimeout 60000000 #define MaxInterfaces 5 int32_t FirstInterface = 1; int32_t LastInterface = 3; int32_t fd[MaxInterfaces]; int32_t ifindex[MaxInterfaces]; int32_t displayLevel = 0; /* displayLevel = 0: no display mode displayLevel = 1: statistics-only mode displayLevel = 2: learning-table dump mode displayLevel = 3: debug mode displayLevel = 4: louder debug mode */ int32_t numLearned = 0; int32_t numInput = 0; int32_t numOutput = 0; extern void _exit(int32_t); time_t time(time_t *timer); /** * Open a socket for the network interface */ int32_t open_socket(int32_t index, int32_t *rifindex) { unsigned char buf[MAX_PACKET_SIZE]; int32_t i; int32_t ifindex; struct ifreq ifr; struct sockaddr_ll sll; unsigned char interface[IFNAMSIZ]; strncpy(interface, "ethX", IFNAMSIZ); interface[3] = '0' + index; int32_t fd = socket(PF_PACKET, SOCK_RAW, htons(ETH_P_ALL)); if (fd == -1) { printf("%s - ", interface); perror("socket"); _exit(1); }; // get interface index memset(&ifr, 0, sizeof(ifr)); strncpy(ifr.ifr_name, interface, IFNAMSIZ); if (ioctl(fd, SIOCGIFINDEX, &ifr) == -1) { printf("%s - ", interface); perror("SIOCGIFINDEX"); _exit(1); }; ifindex = ifr.ifr_ifindex; *rifindex = ifindex; // set promiscuous mode memset(&ifr, 0, sizeof(ifr)); strncpy(ifr.ifr_name, interface, IFNAMSIZ); ioctl(fd, SIOCGIFFLAGS, &ifr); ifr.ifr_flags |= IFF_PROMISC; ioctl(fd, SIOCSIFFLAGS, &ifr); memset(&sll, 0xff, sizeof(sll)); sll.sll_family = AF_PACKET; sll.sll_protocol = htons(ETH_P_ALL); sll.sll_ifindex = ifindex; if (bind(fd, (struct sockaddr *)&sll, sizeof(sll)) == -1) { printf("%s - ", interface); perror("bind"); _exit(1); }; /* flush all received packets. * * raw-socket receives packets from all interfaces * when the socket is not bound to an interface */ do { fd_set fds; struct timeval t; FD_ZERO(&fds); FD_SET(fd, &fds); memset(&t, 0, sizeof(t)); i = select(FD_SETSIZE, &fds, NULL, NULL, &t); if (i > 0) { ssize_t size = recv(fd, buf, MAX_PACKET_SIZE, 0); if (displayLevel == 4 && size > 0) printf("interface %d flushed\n", ifindex); }; } while (i); if (displayLevel >= 4) { printf("%s opened (fd=%d interface=%d)\n", interface, fd, ifindex); } return fd; } /** * Print IPEC packet content */ void printPacket(EtherPacket *packet, ssize_t packetSize, char *message) { printf("%s from x%1x:x%1x to x%1x:x%1x\n", message, ntohs(packet->srcMAC1), ntohl(packet->srcMAC2), ntohs(packet->destMAC1), ntohl(packet->destMAC2)); } /** * Forwarding-table Handler */ /* Forwarding table entry data structure */ struct ForwardingTable { time_t tv_sec; // timestamp 'sec'-part suseconds_t tv_usec; // timestamp 'usec'-part uint32_t idh; uint32_t idl; int32_t ifnum; unsigned short age; } __attribute__((packed)); #define FWDTBLSIZE 1000 struct ForwardingTable fwdtbl[FWDTBLSIZE]; /* Initialize forwarding table */ void initfwdtbl() { int32_t i; for (i = 0; i < FWDTBLSIZE; i++) { fwdtbl[i].idh = 0; fwdtbl[i].idl = 0; } } /* Dump a forwarding table entry */ void dumpfwdtbl(int32_t index, char *message) { printf("%s index=%d id=x%04x%08x ifnum=eth%d age=%d timestamp=%d.%d\n", message, index, (uint16_t)fwdtbl[index].idh, (uint32_t)fwdtbl[index].idl, fwdtbl[index].ifnum, fwdtbl[index].age, (int32_t)fwdtbl[index].tv_sec, (int32_t)fwdtbl[index].tv_usec); } /* Snapshot the forwarding table */ void snapfwdtbl() { int32_t i; printf(" Learned addresses:\n"); for (i = 1; i < FWDTBLSIZE; i++) { if (fwdtbl[i].idh != 0 || fwdtbl[i].idl != 0) { printf(" %2d %04x%08x eth%d\n", i, fwdtbl[i].idh, fwdtbl[i].idl, fwdtbl[i].ifnum); } else { return; } } } /* Search the forwarding table (linear search used currently) */ int32_t getfwdtbl(uint32_t idh, uint32_t idl) { // int32_t getfwdtbl(uint64_t id) { int32_t i; for (i = 1; i < FWDTBLSIZE; i++) { // (fwdtbl[0] not used) if (fwdtbl[i].idh == idh && fwdtbl[i].idl == idl) { // already registered return i; } else if (fwdtbl[i].idh == 0 && fwdtbl[i].idl == 0) { // not yet registered return -i; } } return 0; // Table full! } /* Fill a forwarding table entry */ int32_t setfwdtbl(EtherPacket *packet, int32_t index, int32_t srcIfnum) { struct timeval tv; struct timezone tz; fwdtbl[index].idh = ntohs(packet->srcMAC1); fwdtbl[index].idl = ntohl(packet->srcMAC2); fwdtbl[index].ifnum = srcIfnum; gettimeofday(&tv, &tz); fwdtbl[index].tv_sec = tv.tv_sec; fwdtbl[index].tv_usec = tv.tv_usec; if (displayLevel >= 3) dumpfwdtbl(index, "\nNewly learned:"); return index; } /* Return true iff the forwarding table entry is timed-out. */ int32_t timedOut(struct timeval now, struct ForwardingTable fwdentry, suseconds_t timeoutTime) { double sec = (now.tv_sec - fwdentry.tv_sec) + ((double)(now.tv_usec - fwdentry.tv_usec - timeoutTime)) / 1000000; if (displayLevel >= 4) { printf("%s: now %d.%d fwdtbl %d.%d timeout %d msec %d\n", sec > 0 ? "Timed out" : "Not timed out", (int32_t)now.tv_sec, (int32_t)now.tv_usec, (int32_t)fwdentry.tv_sec, (int32_t)fwdentry.tv_usec, (int32_t)timeoutTime, (int32_t)(1000 * sec)); } return sec > 0; } /** * Forward a packet */ int32_t forward(EtherPacket *packet, ssize_t sizein, int32_t srcIfnum) { struct timeval now; struct timezone tz; gettimeofday(&now, &tz); // get current time int32_t ifnum; if (displayLevel >= 3) { printPacket(packet, sizein, " "); } int32_t index = getfwdtbl(ntohs(packet->srcMAC1), ntohl(packet->srcMAC2)); if (index == 0) { // Table full! _exit(1); } else if (index < 0) { // source not yet learned - learn setfwdtbl(packet, -index, srcIfnum); numLearned++; // 2011-2-23 } else { // already learned if (displayLevel >= 4) dumpfwdtbl(index, "Before:"); if (timedOut(now, fwdtbl[index], LearningTimeout)) { // timeout (_sec?!) if (displayLevel >= 3) { printf("Updated! if: %d -> %d, timestamp: %d.%d -> %d.%d\n", fwdtbl[index].ifnum, srcIfnum, (int32_t)fwdtbl[index].tv_sec, (int32_t)fwdtbl[index].tv_usec, (int32_t)now.tv_sec, (int32_t)now.tv_usec); } fwdtbl[index].ifnum = srcIfnum; fwdtbl[index].tv_sec = now.tv_sec; fwdtbl[index].tv_usec = now.tv_usec; } } struct sockaddr_ll sll; memset(&sll, 0, sizeof(sll)); sll.sll_family = AF_PACKET; sll.sll_protocol = htons(ETH_P_ALL); uint32_t destMAC1 = ntohs(packet->destMAC1); uint32_t destMAC2 = ntohl(packet->destMAC2); int32_t destIndex = getfwdtbl(destMAC1, destMAC2); if (destIndex < 0 || // next-hop interface not known timedOut(now, fwdtbl[destIndex], ReferenceTimeout)) { // timeout (_sec?!) // flood packet if (displayLevel >= 3) { printf("@@@@@@@@\n@ FLOOD packet @ (dest=%x:%x)\n@@@@@@@@\n", destMAC1, destMAC2); } for (ifnum = FirstInterface; ifnum <= LastInterface; ifnum++) { if (srcIfnum != ifnum) { sll.sll_ifindex = ifindex[ifnum]; ssize_t sizeout = sendto(fd[ifnum], packet, sizein, 0, (struct sockaddr *)&sll, sizeof(sll)); if (sizeout < 0) { perror("sendto"); } else { if (displayLevel >= 4) printf("%d bytes sent through eth%d\n", sizeout, ifnum); } } } } else { int32_t destIfnum = fwdtbl[destIndex].ifnum; if (displayLevel >= 3) { printf("@@@@@@@@@\n@ SWITCH packet @ (dest=%x:%x) to eth%d\n@@@@@@@@@\n", destMAC1, destMAC2, destIfnum); } sll.sll_ifindex = ifindex[destIfnum]; ssize_t sizeout = sendto(fd[destIfnum], packet, sizein, 0, (struct sockaddr *)&sll, sizeof(sll)); if (sizeout < 0) { perror("sendto"); } else { if (displayLevel >= 4) printf("%d bytes sent through eth%d\n", sizeout, destIfnum); } } } /** * Main program */ int32_t main(int32_t argc, char **argv) { unsigned char buf[MAX_PACKET_SIZE]; int32_t ifnum; // interface number (different from ifindex) int32_t count = 0; time_t lastTime, timer; if (++count < argc) { FirstInterface = atoi(argv[count]); // Min interface # } if (++count < argc) { LastInterface = atoi(argv[count]); // Max interface # } if (FirstInterface < 0 || LastInterface < FirstInterface || MaxInterfaces < LastInterface) { printf("Interface range (%d..%d) must be between 0..%d\n", FirstInterface, LastInterface, MaxInterfaces); } if (LastInterface < FirstInterface + 2) { printf("Three or more interfaces are required for switching\n"); } if (++count < argc) { // Debug print displayLevel = atoi(argv[count]); } // Open raw sockets and initialize for sending packets for (ifnum = FirstInterface; ifnum <= LastInterface; ifnum++) { fd[ifnum] = open_socket(ifnum, &ifindex[ifnum]); // Set non-blocking mode: int32_t flags = fcntl(fd[ifnum], F_GETFL, 0); fcntl(fd[ifnum], F_SETFL, O_NONBLOCK | flags); } initfwdtbl(); lastTime = time(&timer); // Forwarding operation for (;;) { for (ifnum = FirstInterface; ifnum <= LastInterface; ifnum++) { ssize_t sizein = recv(fd[ifnum], buf, MAX_PACKET_SIZE, 0); if (sizein >= 0) { if (displayLevel >= 3) { printf("\nReceived through eth%d (%d bytes)\n", ifnum, sizein); } forward((EtherPacket*)buf, sizein, ifnum); count++; if (displayLevel >= 1) { time_t now = time(&timer); if (now >= lastTime + 1) { printf("\n\n"); if (numLearned > 2) { printf("Learning: %3d times\n", numLearned); } else if (numLearned == 2) { printf("Learning: Twice\n"); } else { printf("Learning: Once\n"); } if (displayLevel == 2) { snapfwdtbl(); } printf("External packets: RX %4d TX %4d\n", numInput, numOutput); lastTime = now; } } } } } }
The content of Ether.h used in this program is as follows.
/*** * * Software-based Ethernet Common Header * * Coded by Yasusi Kanada * Ver 0.2 2011-1-11 Initial version * ***/ #include <linux/if_packet.h> #include <linux/if_ether.h> #include <net/if.h> #include <sys/ioctl.h> #include <sys/socket.h> #include <sys/time.h> #include <stdio.h> #include <elf.h> #include <string.h> // 0 6 12 14 // +-------+-------+----+---------------+ // | DMAC | SMAC |Type| Payload | // +-------+-------+----+---------------+ struct _EtherHeader { uint16_t destMAC1; uint32_t destMAC2; uint16_t srcMAC1; uint32_t srcMAC2; #ifdef VLAN uint32_t VLANTag; #endif uint16_t type; int32_t payload; } __attribute__((packed)); typedef struct _EtherHeader EtherPacket;]]>
Run the program on a node on the network (slice) in the following method.
./test <VNode#> <#Links>
For example, run it on all the nodes in the network simultaneously. <VNode#> is the node number (Give unique numbers to nodes including terminal nodes), <#Links> is the number of links N. (Assume that eth1 to ethN are used.)
For example, to show the connection between (virtual) nodes 1, 2, 3, each of which has three links, the following command is to be run on each node.
./test 1 3
./test 2 3
./test 3 3
The following message shows the output obtained by running "./test 3 3" (on VNode 3). These messages are repeated. (Each node outputs a packet once in every second to each link.)
Received from VNode 1 through eth1 (56 bytes) Received from VNode 2 through eth3 (56 bytes)
These messages mean that packets received by VNode 3 come from VNode 1 through eth1, and from VNode 2 through eth3.
/*** * * Non-IP Network Connection Tester * * Coded by Yasusi Kanada * Ver 0.2 2011-6-14 Initial version (for Linux promiscuous mode) * Ver 0.8 2012-5-23 Argument count updated * ***/ #include <linux/if_packet.h> #include <linux/if_ether.h> #include <net/if.h> #include <sys/ioctl.h> #include <sys/socket.h> #include <sys/time.h> #include <stdio.h> #include <elf.h> #include <string.h> #include <fcntl.h> #define bool int32_t #define true 1 #define false 0 #define DEBUG 0 #define MAX_PACKET_SIZE 2048 // Sufficiently larger than the MTU #define FirstInterface 1 #define LastInterface 16 int32_t fd[LastInterface]; int32_t ifindex[LastInterface]; int32_t firstInterface; int32_t lastInterface; extern void _exit(int32_t); /** * Open a socket for the network interface */ int32_t open_socket(int32_t index, int32_t *rifindex) { unsigned char buf[MAX_PACKET_SIZE]; int32_t i; int32_t ifindex; struct ifreq ifr; struct sockaddr_ll sll; unsigned char interface[IFNAMSIZ]; strncpy(interface, "ethX", IFNAMSIZ); interface[3] = '0' + index; int32_t fd = socket(PF_PACKET, SOCK_RAW, htons(ETH_P_ALL)); if (fd == -1) { printf("%s - ", interface); perror("socket"); _exit(1); }; // get interface index memset(&ifr, 0, sizeof(ifr)); strncpy(ifr.ifr_name, interface, IFNAMSIZ); if (ioctl(fd, SIOCGIFINDEX, &ifr) == -1) { printf("%s - ", interface); perror("SIOCGIFINDEX"); _exit(1); }; ifindex = ifr.ifr_ifindex; *rifindex = ifindex; // set promiscuous mode memset(&ifr, 0, sizeof(ifr)); strncpy(ifr.ifr_name, interface, IFNAMSIZ); ioctl(fd, SIOCGIFFLAGS, &ifr); ifr.ifr_flags |= IFF_PROMISC; ioctl(fd, SIOCSIFFLAGS, &ifr); memset(&sll, 0xff, sizeof(sll)); sll.sll_family = AF_PACKET; sll.sll_protocol = htons(ETH_P_ALL); sll.sll_ifindex = ifindex; if (bind(fd, (struct sockaddr *)&sll, sizeof(sll)) == -1) { printf("%s - ", interface); perror("bind"); _exit(1); }; /* flush all received packets. * * raw-socket receives packets from all interfaces * when the socket is not bound to an interface */ do { fd_set fds; struct timeval t; FD_ZERO(&fds); FD_SET(fd, &fds); memset(&t, 0, sizeof(t)); i = select(FD_SETSIZE, &fds, NULL, NULL, &t); if (i > 0) { recv(fd, buf, i, 0); }; printf("interface %d flushed\n", ifindex); } while (i); printf("%s opened (fd=%d interface=%d)\n", interface, fd, ifindex); return fd; } /** * Send a packet */ int32_t sendPacket(int vnodeNum) { unsigned char packet[MAX_PACKET_SIZE]; struct sockaddr_ll sll; int ifnum; packet[0] = vnodeNum; memset(&sll, 0, sizeof(sll)); sll.sll_family = AF_PACKET; sll.sll_protocol = htons(ETH_P_ALL); for (ifnum = firstInterface; ifnum <= lastInterface; ifnum++) { int sizein = 14; // >= ethernet packet size ! sll.sll_ifindex = ifindex[ifnum]; if (DEBUG) { printf("Send to eth%d\n", ifnum); } ssize_t sizeout = sendto(fd[ifnum], packet, sizein, 0, (struct sockaddr *)&sll, sizeof(sll)); if (sizeout < 0) { perror("sendto"); } } } /** * Main program */ int32_t main(int32_t argc, char **argv) { unsigned char packet[MAX_PACKET_SIZE]; int32_t ifnum;// interface number (different from ifindex) int32_t vnodeNum = 1; struct timeval now; struct timezone tz; gettimeofday(&now, &tz); time_t time = now.tv_sec; int32_t count = 0; if (++count < argc) { vnodeNum = atoi(argv[count]); } firstInterface = FirstInterface; lastInterface = LastInterface; if (++count < argc) { lastInterface = atoi(argv[count]); } // Open raw sockets and initialize for sending packets for (ifnum = firstInterface; ifnum <= lastInterface; ifnum++) { fd[ifnum] = open_socket(ifnum, &ifindex[ifnum]); // Set non-blocking mode: int32_t flags = fcntl(fd[ifnum], F_GETFL, 0); fcntl(fd[ifnum], F_SETFL, O_NONBLOCK | flags); } for (;;) { for (ifnum = firstInterface; ifnum <= lastInterface; ifnum++) { ssize_t sizein = recv(fd[ifnum], packet, MAX_PACKET_SIZE, 0); int senderNum = packet[0]; if (sizein >= 0) { printf("Packet received from VNode %d through eth%d (%d bytes)\n", senderNum, ifnum, sizein); } } gettimeofday(&now, &tz); time_t time1 = now.tv_sec; if (time1 > time) { sendPacket(vnodeNum); time = time1; } } }]]>