Interesting Networking Issues reported by customer and Important simple things to remember..!!!

 Having around 15 years of experience in networking products like Switch/Router, GPON Access devices I got to work on several features on these products across different companies that include Brocade, Nokia, Matisse Networks, 3Com products. While I moved to Core networking after a brief stint in Telecom, I worked on several customer issues some of which were quite interesting, some easy and some very tedious. 

Each of the issues I worked on gave me a lot of lessons to learn. Here I present some of the interesting customer issues I worked on. But before I jump in to it, let me tell you in most of the times the customer reports only a symptom and the R&D has to start from there to identify the actual root cause. In most of the difficult scenario's, the issue WILL NOT be reproducible and so the R&D team has to grab the information as much as possible before cornering the root cause. 

For starters, when I mention the customer reports only a symptom, it literally means he tells about what he sees. Some of the examples are : "I executed the command and system crashed." Chances are the system could crash due to other commands as well if it is running on low memory, but customer will not be aware of it. 

Another example: "Packets are dropping or connections are timeout." They will not tell if they made any configuration changes or if any HW was changed or any connections were changed. it is the R&D job to ask as many questions as possible to get the answers. Following things are very important things to collect: 

1. In case of crash, the stack trace left behind. In Linux based systems, a coredump would be valuable. 

2. Syslog data will be crucial to identify what the customer has executed during the time of issue. 

3. observing timestamps is of utmost importance. In fact, this is the first thing to do. Check the timestamps of the issue and match it with syslog. In some cases, it saves lot of time. 

4. Non default configuration present in the system and the last command executed before the issue cropped up. 

5. Customer DB usually is not shared unless it becomes important. Get it if possible for easy analysis.

6. Depending on the kind of problem reported, relevant Debug commands related to the problem need to be collected and shared with the TEC/TAC team which interacts with customer. They need to be handled with lot of caution as any wrong execution on the device can create other problems too. 

7. It is better for R&D to avoid direct interaction with the customer and let TEC/TAC front end. Most of the cases, by the time the issue is in R&D lap, there would have been lot of time lapsed and this can irate the customer. Their frustration can spill on and it is better TEC/TAC handle it as they are trained to deal with it. Women deal these scenarios in much better way. 

7. Ensure there is a demo of debug activity done before facing the customer. 

8. Gather the kind of terminals customer is using as well. 

So, moving to the issues I faced, the title is the customer described symptom and the description has the steps we followed to Root cause it. Chances are you may end up surprised as to where the issue began and where it ends !!! 

1. Issue: Executing a Show command in SSH is causing the CLI crash, but not seen when using TELNET: 

Usually when such crashes are reported, it should be a cakewalk for the R&D as the stack trace should point at the location where it crashes. Relevant code walk through would have helped. But in this case, the same command does not crash when executed through TELNET. This means, there is no apparent problem with the functionality that is being run. So the suspicion was on SSH protocol stack. When the stack trace was seen it was pointing at code which was related to array. When we tried to reproduce using the same command using SSH it was working fine. One clue we got from the trace was that the problem was related to the window size aspect of the display. But we are not able to know what was the problem. We had a lucky breakthrough when the customer shared how the output looked like when the issue happened. We noticed there were abnormally large number of headers like "---------" before the columns were published. Normally we see it followed immediately with the column names. This made me wondering what terminal customer was using like putty, tera term, xterm etc. I asked TEC team to find out how the customer is running the command, what are its settings and name of the terminal so we can use the same. 

When the TEC approached the customer, he gave a demo of their terminal and the TEC observed one interesting setting where the window size was mentioned as -1. This was sufficient for me to identify the problem. 

Whenever SSH is done, as part of the transactions, size of the window is also exchanged so as to align the display. There is no limit as such for it other than sizeof integer. This limit is however set to 1024 in TELNET in server. When customer is using the terminal with -1 setting, the size of window is received as 0xFFFFFFFF and SSH was trying to allocate this large memory and failing. The failure is not handled correctly in code as this part was always deemed to be success!!! 

Now that we understood the cause, we were able to generate the crash by using a script that starts session with window as high as 2000000. 

The fix is anybody's guess ..!! 

Lesson 1: Each CLI Application terminal brings its own features and limitations. 

2. Starting a download through SNMP causes Router to reboot (crash): 

This issue started with a simple notion that the stack trace would be sufficient to identify the fix. Since it is SNMP related, the SNMP interfaces will call some API that would initiate a download. As is the case with crashes, the last API in the trace was suspected as it is deemed to have caused it. But then came the surprise. The last API in the trace does not seem to be related with the purpose of the object which was used for initiating the download. it is like if we are initiating a download for one file, the action seen was for a download of totally different application. 

I started going back following the trace and then arrived at the SNMP API that was supposed to have started it all. usually when we are less experienced, we tend to believe the code already in the system is well tested and the code is expected to work. But it is not the case. There will be legacy code which undergoes transformation with time and in some cases, the old API's cease to have their relevance. 

Similar thing happened. The Interface being called from SNMP set code is calling an Old API which was no longer applicable or useful for this application. I immediately checked with R&D team to understand why they continued to use it. They were equally surprised, and agreed to change it to the more useful API that helped fix the issue. 

Lesson 2: It is equally important to suspect the first API as the last. Suspect the Legacy..!!

3. Ports are removed from All VLAN's including default VLAN 1 ..!!!!!!!!!!

How can a port be not part of any VLAN? At least it should be part of default VLAN.. isn't it? This issue was both surprising and annoying for the customer. There was lot of packet loss before they realized this behavior and had almost the entire managers lined up to explain why it entered this state. 

When they were queried as to what did they do, they just said "we do not know what happened". We configured few modules and then noticed it. 

The customer facing TEC team were deployed to reproduce the issue. with little information, they never able to reproduce it. One of my team mates who was working on it offered to add some debug messages and share the build to the customer for them to repeat the same process in the lab. This was done with a hope the debug messages would give some hint. 

I was assigned after he went for PTO and by that time it was already having attention of Director of Sustenance engineering. I went through the syslog file and checked the logs and also the debug prints shared by the TEC team and found there was a TFTP based configuration download that has happened. 

So I tried the same with some combination initially with little luck. Finally I hit the condition of port not being part of any VLAN but I did not know how I entered the state !!!. Again I had to recollect the combination of things I tried. I had to reboot the system restore it and started all again. 

somehow I remembered I downloaded the configuration from TFTP and after some manual configurations, tried it. I was lucky and hit the issue. 

Then I realized the problem is seen only when configuration is being downloaded through TFTP. I started reducing the configuration and checked as to which command was causing the problem. 

Finally I zeroed in on the "end" command!!!. 

The root cause was due to the CLI architecture that router had. Normally in a CLI configuration, to move from one mode to previous mode, there is a exit or end command depending on products. The end command will drive the CLI mode from wherever it is to the root node. In this process, the CLI applies/sets the mode and stores all the command nodes that are available in that mode. 

In One of these commands that were executed, there was an erroneous code that does memset to 0 a portvlan bitmask membership which caused all the VLAN's to be removed from the port. 

Once we were able to identify this code, fix was just a LINE...!!!!

Lesson 3: Write down the set of actions you are performing when you are trying to reproduce an issue. You never know when you get lucky..!!

More to come ...!!!!

Holiday in Kanyakumari and Trivandrum..!

We had been to a recent trip to Kanyakumari and Trivandrum through IRCTC package and following is our experience which I thought of sharing so that it could be useful for anybody's planning.
Following is the link to IRCTC package which we took:
http://www.irctctourism.com/TourPackages/RailTour/Chennai-Kanyakumari-Trivandrum-Tour.html

Following are the pros and cons of this package:
1. Since few tickets (4-6) are blocked for this package and so late planning (just around 4 days ahead) might work if they are available.
2. All we need to do is pay the money and they will take care of the tour.

However, here are the issues we faced:
1. Hotel Rooms are available from 12:00PM and so a temporary room is allotted (may be due to year end rush, we went during 30/12).
2. In Kanyakumari, no complimentary breakfast.
3. The tour operator seem to be focused on getting a minimum 4-5k profit for trip and so cut the costs.
4. Hotel booking in our case was done in someone else name and we had some confusion. After speaking to IRCTC contact, it got resolved. Not even Water bottle was offered in the room. But the hotel was fine, sea facing. We got in to Singaar international for a day's stay in Kanyakumari.
5. Tour operator in KK, Trivandrum, stick to the itinerary and so if there are any other things to see, they won't help.
6. Since we opted out of seeing Wax museum (already we saw in SFO, CA, USA), the driver took us to beach at 4.00 PM itself for sun set. (Weather was very hot).
7. Driver is from Core TN and so do not expect him to speak any other language. Our driver was a 23 year old and his communication skills were extremely poor. He just did not have customer orientation as he was only 20 days in to the job.
8. We met other customers on the tour and they too had similar issues. Their car was not good. At least this complaint we did not have.

In Kanyakumari, every important site was nearby. Hotel is near to Railway station. We do not need the package practically, since the bus stop is also near. 

Things we did on the tour:
1. Started on 29th Dec. Boarded Kanyakumari Express from Egmore at 5.30 PM. We took comfort package so got good tickets 41, 42. Reached kanyakumari next day.
2. Hotel guys gave temporary room as the checkin is only after 12.00PM. Got ready and had Breakfast buffet (Rs 175 per head).
3. Driver dropped us near the temple compound and told everything is nearby and left.
4. We unfortunately did not prefer standing in Line for ticket for Vivekananda memorial which already had a km queue. So went for Bhagawathy amman temple.
5. Darshan was smooth.  There was a 20 Rs ticket for quick darshan. Temple closes at 12.00 PM.
6. By the time we are out, the line for memorial was 4km long and so we went to the rear part of the temple where there are other things to look at i.e. Three seas merging point, Adi sankara temple, Gandhi mandapam.
7. Spent some time near three seas merging point and did some shopping.
8. There are some cheap items for Home interior decorations like wall hangings and plastic flowers. Hangings made of sea shells we got a dozen for 120. Here in mahabalipuram, it was 30Rs per one hanging. The hanging is around 6ft. problem is they do not accept credit cards.
9. Got back to Hotel and had lunch. Took rest for an hr, and then driver picked us.
10. We thought we will be taken to some far off place for sight seeing. But he took us to Wax museum which was walkable from the hotel. We opted not to go as we saw a better museum before.
11. He took us to Sun set point which was 30 min away. it was a beach again. We would have enjoyed had it been little late in the evening. But still, the beach was good. One can go deep in to the water.
12. Came back after sunset and took rest in the hotel.
13. Next day, early morning, we checked out and driver took us to suchindram temple.
14. Followed it with a boat ride in Neyyar backwaters on the way to Kovalem. 
15. Ride was priced at 3500 for 2 people for 1.30 min. for each additional person, 500 Rs. Midway of the ride along the waters, there are floating restaurants where we can get lunch for fair price. Non vegetarians can enjoy much. Rice was basmati and food was reasonably good. 
16. Proceeded to Kovalem and had a nice fast motor boat ride. For a 20 min tour around the sea, we paid 1500 rs. It would be advantageous if more people are there since the charge is per ride and not per person. Ride was good. But only for the people who can dare it.
17. Checked in to KTDC hotel. This hotel is just opposite to the Trivandrum railway station and Bus stand. Old hotel. Offered complementary breakfast.
18. Did some shopping along the MG road and took rest for the night. AC was not controllable.
19. Early morning started to Padmanabha swamy temple. Unfortunately, it was closed for Nivedyam at 7.00 AM and opened at only 8.30 AM. Following are the timings.

Morning

03.30 am to 04.45 am (Nirmalya Darshanam) 06.30 am to 07.00 am 08.30 am to 10.00 am 10.30 am to 11.10 am 11.45 am to 12.00 Noon

Evening

05.00 pm to 06.15 pm 06.45 pm to 07.20 pm

Pooja cost is 180 or 150 for 2 people. You will get prasadam free :-)
Only Dhoti is allowed for Men. Women should wear saree. If not, they have to tie a dhoti around. Girls should not wear any western wear. Completely ethnic wear. Please check the site for more details: http://www.sreepadmanabhaswamytemple.org/

20. After darshan, had breakfast and checked out of the room. Went to Kanakannu palace which was closed on Sunday. (IRCTC planning really sucks here). Nearby there is Napier museum which has antiques since Xth Century.

21. Our return journey was from nagarcoil which is 100 KM away from Trivandrum. Had to waste 3 Hrs just to catch train back. Return train is also Kanyakumari Express which starts at 5.30 PM.

Had IRCTC planned the return journey from Trivandrum, it would have been good.  On the way back we did not have time for lunch. Picked up some snacks and started back to Chennai.

Overall, the package is really not so worth. If we take our own cabs we can go to any place and see it.
if you are planning in advance, better do it yourself. Everything is nearby. Just go to kanyakumari, hire pvt cabs or if comfortable, travel by buses.

Interview Questions: Interview FAQ's I faced

Interview Questions: Interview FAQ's I faced: "My background was SNMP, L2 technologies. So most of the questions were related to it. Of course C too. Company : Cisco (interview : 5 hrs)..."

Interview FAQ's I faced

Interview questions from Cisco, Brocade, Juniper, Broadcom, Aricent, HCL.

Company : Cisco (interview : 5 hrs)
1. Program for String reversal in best efficient way.
2. Program for reversal of n characters in a string. Say string is "interview", your o/p should be "tinvrewei". Reversing for every 3 characters.
3. Basic working of Switch.
4. Use of PVID.
5. Ingress filtering use.
6. ACL's and their use. What is TCAM.
7. Difference between process and threads.
8. How a thread is a light weight process.
9. Difference between binary and counting semaphores. How binary sem is different from mutex.
10. Difference between SNMP V1, V2,V3 protocols.
11. What is Gauge used for. DIfference between Gauge and Counter.
12. Explain what happens when www.yahoo.com is entered in the browser. All the communication that happens. How will the data be retrieved. What happens if the DNS server is connected to a different network.
13. Compose the datastructure that is used in SNMP while searching for a particular OID. Explain the entire search process done to ascertain the OID.
14. give examples for RTOS applications. Where and why they are preferred. How the latency is reduced. Advantages of RTOS over other OS.
15. If a faulty thread is present which has infinite loop, explain how the scheduler handles it. How to fix this ?
16. How to determine merging in a linked list.
17. How to use two queue's to get the behavior of a stack.
18. What are the traversals available. Come out with a traversal technique which can print 1,2,3,4,5,6,7,8 for a tree shown below.
                   1
           2               3
     4         5      6     7
  8
19. How do you search for duplicate data in a linked list?
20. How to know if a data structure was modified when it is shared between different threads which can read, modify, delete data from it. What mechanism is followed in general.
21. How to debug a system hang.
22. How to debug a crash. Especially when the core file does not give proper information.
23. Priority inheritence explain.
24. What are the IPC mechanisms available.
25. How will the scheduler schedule the threads in a normal OS (i.e. Non premptive priority based). How will it handle when a thread malfunctions in the process.
26. Explain how security is achieved using SNMP V3.
27. How  do you determine Little endian or Big Endian. Write program.
28. Write prog for ntohs()
29. Explain STP, RSTP and MSTP. Which one is better top down or bottom top model for ST, w.r.t Root bridge position. Edge ports use.
30. How to detect a memory is corrupted? What are the mechanisms used?
31. What are the different signals you have used?
32. what are the different segments you know.
33. what is a heap data structure.
34. What happens if a ARP packet is L3 broadcast?
35. How do you implement a timer library.

Company Juniper: (interview : 5hrs)
1. Use of volatile keyword.
2. How to delete a data in a linked list using only single pointer.
3. Explain switching process.
4. What are the contents of table used for switching.
5.  Shared Vlans MAC Vlan table example.
6. Ingress filtering.
7. difference between L2 Broadcast and L3.
8. Compose a data structure for Semaphore.
9. Ntohl implementation.
10. given an array of 1's and 0's in random order, how will u arrange them in such a way that 1's are on one side and 0's on other.
11. How fragments are done? Where will they be joined ? Explain why?  Given 3 routers with different MSS.
12. Difference between binary and Mutex. Functions used .
13. ARP which layer protocol ? How is it useful.
14. 8 Queens problem
15. Basic switching explanation.
16. Size of structure without using arrays and sizeof.
17. What is hard time and soft time?
18. Use of Volatile?
19. Explain communication if a host is connected to a switch and then the gateway while doing a ping?

Company : Brocade ( interview : 1 .5 hrs telephonic)
1. Difference between trap and Inform.
2. Enhancements done in SNMP V2 over V1. V3 over V2 and V1.
3. How Security is achieved in SNMP V3.
4. Different types of error messages sent from agent.
5. Given three indices of string type, explain the order of Get-Next command output.
6. How are traps received by a manager.
7. What is the difference between counter and gauge.
8. What are the IPC mechanisms used by SNMP in a process based model
9. What are the ports used by SNMP.
10. Is it possible to do a SNMP V3 GET on  an agent that has been operating since 2 days even though no action was performed on the agent continuously.
11. Use of SNMP Engine ID.
12. few questions about MIB generation, Code generation to be used in the image.
13. What are the default traps available.
14. What are the contents of a trap.
15. What is a proxy agent.
16. Test tools used like Silver Creek, and questions relevant to them.
17. What are the initial messages exchanged between Manager and the Agent in a SNMP V3.
18. How are the counters populated and retrieved by the Agent.
19. Which is better, process based model or thread based model. Which is preferred when.
20. Why is it that only one port is used while tx and Rx in a LAG when the src mac address is the same. How do you address reordering problem in LAG. What if any specific frame is lost. Which layer handles loss of frames.
21. Difference between TCAM and normal CAM. What is advantageous. 
22. What are the contents of trap?

Company: Broadcom. (interview : 6 Hrs)
1. *p++ what does it do when assigned to any variable.
2. const char *p ,char *const p , const * char p differences.
3. inserting node in the middle of a linked list.
4. What is the difference between STP, RSTP, MSTP. Explain how the convergence is achieved in RSTP. What are the port states that are present in each of the versions.
5. Explain LAG. Why it is not supported on a half duplex port. What is a static LAG.
6. Given two images used intermittently in a single RAM, how do you ensure proper communication between the two images in case each image wants to share some information with other.
7. What is alignment for a structure node { char c1;int n1;char c2;int n2;}. How can you avoid alignment problems.
8. What is the difference in behavior between ingress filtering enabled and disabled. How will it behave for tagged and untagged frames received on a port.
9. What is an ACL, TCAM. How are the rules prioritized in ACL. What are the actions available. What will the TCAM do in case there are multiple similar rules. Will it check for all the protocols configured for denial or accept?
10. What are the IPC mechanisms in Linux.
11. Design process done in the company. Testing methodologies used.
12. What is the relation between STP and LACP. Any specific scenario?
13.What is QinQ? how is it used and where is it used
14. Explain all the process involved in retrieving the webpage yahoo.com
15. List all the protocols that are used in the above process.
16. what are SAP's in networking where are they?
17. What does a ping do? will it call the ICMP application ?

Company : Aricent (interview : 2hrs)
1. Frame format of Dot1Q. Use of TPID. What if the vlan id is 0? How does it classify.
2. How to see the contents of an archive file?
3. Explain STP, RSTP, MSTP differences.
4. Explain Link OAM and how it can be used to detect faults.
5. How is the Discovery process done in Link OAM. How are the errors notified to the customer? How can the loopback detect the problems. How the configuration is done. is there any default configuration that can be used?
6. const char *p and char *const p differences.
7. What is segmentation fault? which signal is used to indicate it.
8. What is volatile used for. What is the difference between volatile const char *p and const char *volatile p.
9. What is QinQ? how is it used and where is it used
10. Ingress filtering related questions.
11. A TCP socket is used in sendto(). Will the packet pass?
12. What happens when a fork is done. How will the scheduler adjust the time slice for it?
13. What are the different system calls that change the execution from user mode to kernel mode?
14. How is process different from a thread.
15. DIfference between Mutex and Binary semaphore?
16. Different IPC mechanisms used/available.
17. What is a zombie process?
18. What is padding problem? How can it be controlled? How is it handled using pragmas?
19. SNMP qns that are mentioned in Brocade were asked.
20. o/p of the program
 int main()
{
int *p;
p=value();
printf("%d",*p);
return 0;
}
int * value()
{
int p=100;
return &p;
}
21. What does a Malloc do. Return type of malloc.
22. Declare a function pointer.
23. Application of counting semaphore.
24. what are ACL's used for?
25. What is the ethertype for QinQ, VLAN.
26. What is the use of ioctl?
27. how are the sockets used for a communication link.
28. What is internal fragmentation and external fragmentation.
29. What is the difference between shared object and a library?
30. Use of Static key word. What are the different storage classifiers available.
31. Use of extern key word.
32. Difference between typedef and macro.

Company : HCL (interview time : 2hrs)
1. Write a macro to find maximum of two numbers. enhance it for 3 numbers.
2. Debugging command to add breakpoint.
3. How to debug a thread in gdb.
4. How to attach a core file in gdb.
5. What is priority inversion ? explain.
6. Tasks A,B,C are having priorities 100, 200, 255 respectively. Task C is holding a semaphore. Assuming priority inheritance is in place, if Task C wants a resource held by Task B, will it get the resource? if so what would be the priority of each task for that to be achieved. If not why?
7. How will free know how many bytes of memory to be freed? How is the sanity check implemented.
8. Write a program to delete a node in a Linked list.
9. Explain the timer library implementation, for a granularity of 1 sec given the efficiency should be only by time.
10. How will the interface number on which the response has to be sent is known to the SNMP Agent once it needs to send a reply?
11. Port numbers used for SNMP. What is inform?
12. Difference in SNMP v2 trap and v1 trap.
13. Questions related to VLAN, STP, RSTP, MSTP, LAG, ACL as discussed above in other companies.
14. How the Multicast Mac address is mapped from multicast IP address.
15. How are the control packets identified?
16. Ether type of VLAN, and QinQ. What is QinQ used for.
17. Use of Volatile keyword?
18. How a multicast communication happen. 
19. Declare a function pointer. What is the difference between (void *func)() and void (*func)().

Some Generic questions ( this keeps growing):
1. Use of register Keyword. Can it be declared globally.
2. Can a Macro be declared with variable arguments ? if so How?
3. Can a Nested function be declared? How?
4. Can a Macro with a goto statement and a label be defined? If so, what happens, if the same macro is called repeatedly thrice?

My Friends Interview Questions:

Brocade interview details for IP group
----------------------------------------
Round-1: Basically it's around L2 data path
------------------------------------------
1. What is the difference between switching and routing
2. What is the difference between switch and bridge
3. Explain VLAN classification
4. Explain about priority tagging
5. How VLAN is classified for priority tagged frames
6. An admin is observin that A MAC entry is getting toggled, what is    the problem and what is the solution(loop & STP)
7. How many STP PDUs are sent on a LAG?
8. Why LAG members are always of equal speed? [Other distribution will be
   unequal. And this is ASIC & usage problem. Nothing to do with LACP    protocol]
9. Why we need LAG hashing? What will happen if we don't have hashing?
   And why same session data is transferred always on same LAG member?
10. What is q-in-q. Why we need that?
11. What is VLAN translation and how it helps?
Then He came to "C" and "os" Qs
1. How you reverse a given statement?
2. How you can find duplicates in a given list?
3. How you trace stack fatals
That's all First round is done. :)
Round-2: It's more of design specific interview
------------------------------------------------
He asked only few questions, based on which arguments went in
1. What is q-in-q and why we need q-in-q
2. What all you consider while adding a port to a VLAN
3. What all you consider while adding an ACL to a port
4. What is 802.3ah and why we need that?
5. What is remote loop back, and how we achieve this and what for it is?
Then He came to "C" and "os" Qs
1. How you reverse a given statement? (Same old Q)
2. What is priority inversion?
That's all Second round is also done. :)
Round-3: It's more of protocol implementation
------------------------------------------------
1. What is STP and why we need that?
2. How we achive MSTP.
3. What is the impact of dynamic VLANs on MSTP?
3. HOW LAG and STP works?
4. Can a port be member of multiple LAGS?
 Third round is also done. :). Call is from USA. So he believed that
 I know thing well. ;)
Round-4: It's more of Managerial implementation (F2F in B'lore)
-------------------------------------------------------------
The call is with VP-1
1. What is conflict and how u resolve it? This point went for 30-min discussion.
2. How you can map your work to revenues? (How?)
He asked abt my pay at the end, and then started struggle. :(
Round-5: It's more of Managerial implementation (F2F in B'lore)
----------------------------------------------------------------
The call is with VP-2
1. How good u r at design?
2. Tell me about ur recenet work's design
3. Tell me the design part of malloc and free
4. Tell me why STP and why not STP?
5. Tell me about conflict? (Seems so much of conflicts are around ;))
6. Tell why you need that much salary? (I have many answers ;)
That's all then asked for a written test.
Round-6: Written test
-----------------------
It has 2 questions with it. It's 90-min span stuff.
1. Implemented a linked list with sorted insert
2. Write code for ls -t option.
And Then final Round: Round 7
---------------------------------
1. Explain packet flow path in the switch (All stages)
2. Explain packet flow path in the stack
3. Explain packet flow path in the chassis
That's all, done. Have fun!


Suggested Reading for Networking:
All New Switch Book.
For OS: Understanding Linux kernel 3rd Edition
Understanding Linux Network internals by OReilly Publishers.

The following link may be of help to you for some of the questions mentioned above.
http://profile.iiita.ac.in/pkmallick_03/adfaqpublish.html