[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: Porting and Extending a TCP/IP noise simulator from kernel 2.2.x to 2.4.x





> -----Original Message-----
> From: Shlomi Fish [mailto:shlomif@techst02.technion.ac.il]
> Sent: Thursday, July 19, 2001 10:41 AM
> To: Shahar Dag
> Cc: linux-il@linux.org.il
> Subject: Re: Porting and Extending a TCP/IP noise simulator 
> from kernel
> 2.2.x to 2.4.x
> 
> 
> On Thu, 19 Jul 2001, Shahar Dag wrote:
> 
> > Hi
> >
> > I know that Linux is not my experty, so maybe my question is silly.
> >
> > >From software engineering point of view creating non 
> official kernel version
> > ==> whenever the kernel updated you loose your work. So it 
> seems as the
> > wrong method.
> > Don't it be better to write a TCP/IP demon/driver and by 
> this be (almost)
> > independent at the kernel version
> >
> 
> FOA, this daemon has to communicate with the kernel somehow, becaue in
> Linux it is the kernel that manages the TCP/IP stack. 
> Secondly, putting
> the arbitration in a module will create a lot of overhead and 
> processing
> delays, which is something we usually want to avoid if we want to
> simulate noise in high load networks (e.g: Gigabit Ethernet).

This statement is wrong - after module is initialised, it behaves just like 
the internal code, as far as you do not want to modify the main packet
dispatcher.

BTW: Are you absolutely sure that you want to simulate noise in GB ethenet?
Think again:
card itself should drop all noisy frames, and thus sk_buffs that come from
giga card are 
marked as though there is no need to checksum those. By applying noise on
DataLink layer
you may cause confusion and despair in TCP stack.

> Regards,
> 
> 	Shlomi Fish
> 
> > Shahar
> > ----- Original Message -----
> > From: "Shlomi Fish" <shlomif@techst02.technion.ac.il>
> > To: "guy keren" <choo@actcom.co.il>
> > Cc: <linux-il@linux.org.il>
> > Sent: Thursday, July 19, 2001 5:14 AM
> > Subject: Re: Porting and Extending a TCP/IP noise simulator 
> from kernel
> > 2.2.x to 2.4.x
> >
> >
> > > On Thu, 19 Jul 2001, guy keren wrote:
> > >
> > > >
> > > > forgive me for this anti-enthusiastic message, but you 
> can implement
> > your
> > > > noise simulator without writing a single line of kernel 
> source - you can
> > > > grab traffic using firewalling rules from kernel mode 
> out into user
> > mode,
> > > > and there u can decide whether to let a packet go on, 
> drop it, send it
> > > > with an error inside, and anything else that a C 
> program allows you to
> > do.
> > > >
> > > > and it'll be able to sustain an ammount of trafic good 
> enough for your
> > > > needs, i imagine. the question, however, is whether 
> such an approach
> > > > doesn't defeat the purpose of the lab.
> > > >
> > >
> > > The project is defined to be in the kernel. Otherwise, I 
> think it won't
> > > be able to scale too much. But maybe we can use the 
> firewalling framework
> > > in order to better simulate the noise. Hmmm....
> > >
> > > > > 1. Is the Networking and TCP/IP stack of kernel 2.4.x 
> very different
> > > > > internally from that of kernel 2.2.x? From what I 
> understood it was
> > > > > modified to support multi-threaded processes, but I'm 
> more interested
> > to
> > > > > know if thecode remained pretty much the same.
> > > >
> > > > some parts were completely rewritten (e.g. the 
> firewalling code). i
> > think
> > > > most changes that got to do with what u call 
> 'multi-threaded processes'
> > is
> > > > actually not related to threads specifically - but to 
> paralel execution
> > in
> > > > general - and got to do with having mroe fine-grained 
> usage of locks
> > > > inside the kernel, to allow taking better advantage of 
> existance of
> > > > seveval CPUs in SMP mode.
> > > >
> > >
> > > No, the problem was that in kernel 2.2.x the TCP/IP stack was not
> > > multi-threaded. That is a blocking accept() call would 
> have blocked the
> > > entire process, and if one thread sent data the other 
> would get stuck,
> > > etc. etc.
> > >
> > > AlexSh can testify on a case where it became apparent.
> > >
> > > > >2. What I thought was to implement some of the noise 
> logic as a patch
> > to
> > > > > the kernel, while implementing the arbitrator of it 
> as a separate
> > module.
> > > > > During the previous project, the whole thing was 
> implemented as apatch
> > > > > and they had to reboot constantly.
> > > > >
> > > > > My question is: can the logic that decides whether to 
> drop, ruin, or
> > how
> > > > > much time to delay a packet, can be implemented as a 
> separate module
> > that
> > > > > will communicate with the main kernel somehow?
> > > >
> > > > yes. you could write your own 'firewall' module, and 
> plug it into the
> > > > kernel. the internel API in kernel 2.4 supports 
> plugging of several API
> > > > modules, chained on-top of each other. this way, you 
> don't need to
> > modify
> > > > the kernel's code at all, and write everything as a module.
> > > >
> > >
> > > If you say so - it might save us a lot of work and we may 
> not need to use
> > > the original code at all (except as a refernce).
> > >
> > > > > 3. I'd like to use kernel debuggers like SGI's kdb 
> and Personal ICE
> > > > > (http://pice.sourceforge.net/) to aid in analysing 
> errors. I know they
> > are
> > > > > not distributed with the kernel of Linus, but I 
> suppose they can be
> > > > > applied quite easily.
> > > > >
> > > > > Assuming the main kernel (which has both or one of 
> the debuggers
> > compiled
> > > > > into it) has an OOPS fault, will they allow me to diagnose it?
> > > >
> > > > you don't need a debugger in order to get an OOPS 
> message diagnosed.
> > klogd
> > > > does that for you, if you maintain an up-to-date symbol 
> map. it'll
> > > > translate the stack into kernel symbols offsets, and 
> you could locate
> > the
> > > > relevant instruction by dissassembling your module.
> > > >
> > >
> > > Yes, but kernel debuggers enable me to see the value of 
> my variables. (!)
> > > and they are interactive and I can do more at that point 
> than just reboot
> > > a safe kernel and look at my source code. At least that's 
> how it is in
> > > Soft-ICE, which Personal ICE aims to resemble.
> > >
> > > > > 4. Naturally, to test the module, I'll need to 
> simulate network
> > activity. I
> > > > > can do that with actual packets that arrive from 
> different machine (or
> > > > > from two different Ethernet cards on the same 
> computer - %-)) but I
> > could
> > > > > also build an internal virtual network, and do some 
> manipulations
> > there.
> > > > >
> > > > > Or could I? I think depending on packets that arrive 
> from the hardware
> > at
> > > > > the early stages of the project is a bad idea<tm>, 
> because kernel
> > > > > programming is bad enough without messing with the hardware.
> > Naturally, we
> > > > > will make sure it works with various hardware, but 
> can we postpone it
> > to
> > > > > later stages? Should we do that?
> > > >
> > > > you could write your code so that you could compile it 
> both in user mode
> > > > and in kernel mode. then you could write the logics in 
> user mode and
> > test
> > > > it there, and only when its debugged - put it in the 
> kernel. i could
> > send
> > > > you the syscall tracker module's sources as an example 
> of one way this
> > > > could be done.
> > > >
> > >
> > > Sounds good. If you can, please put the sys-call tracker 
> source on the web
> > > so my partner and my project's supervisor will be able to 
> look at it.
> > >
> > > I think gdb still beats Personal ICE hands-on, but the 
> kernel debuggers
> > > will probably prove to be useful too.
> > >
> > > > > 5. Simulating network noise may be of interst to Comp. Net.
> > researchers
> > > > > and Net-Admins world-wide. For instance: can MOSIX or 
> Beowulf work
> > with
> > > > > this patch installed on one or more of the clusers' 
> computers? I'd
> > like to
> > > > > find out.
> > > > >
> > > > > So, what do you recommend regarding the issue of 
> whether or not it
> > should
> > > > > be integrated into the main kernel? I think the patch 
> to the kernel
> > should
> > > > > be #ifdef'd because delaying, ruining, or dropping packets
> > deliberately is
> > > > > something most people don't want to enable.
> > > > >
> > > > > Now, Linus objected to the MOSIX code be integrated 
> into the kernel
> > that
> > > > > way, so I don't know if he'll accept such patch. I 
> guess I'll have to
> > ask
> > > > > him. Of course, the patch can be maintained vis-a-vis 
> with the kernel
> > > > > source (like MOSIX), but it would be a better idea if 
> it is in the
> > Linus'
> > > > > source tree.
> > > >
> > > > since it can be done in user mode, or as a kernel module without
> > modifying
> > > > any line of code in the kernel, there's no point in 
> integrating this
> > into
> > > > the kernel's sources.
> > > >
> > >
> > > If we indeed end up with a user-level/kernel module 
> arbitrator, then we
> > > can indeed distribute and maintain it separately from the 
> kernel. However,
> > > if many researchers and Internet workers find it useful, 
> Linus may decide
> > > to include it there.
> > >
> > > > > 6.
> > > > > Do you recommend joining or monitoring the 
> linux-kernel mailing list.
> > I
> > > > > know the amount of daily messages there is very big, 
> much more than
> > all
> > > > > the other mailing-list I am a member of. I could 
> request a bigger
> > quota to
> > > > > accomodate for it, but I'll probably press the "D" 
> button maniacly.
> > > >
> > > > i think i see muli waving his hand enthusiastically 
> with a 'yes' vote :)
> > > >
> > >
> > > I could put it in a separate directory. Put the messages 
> of every week (or
> > > every day?) in a separate mailbox, and maybe get a 
> separate account.
> > >
> > > > > There are various web resources that summarize the 
> activity there. I
> > read
> > > > > the Linux Weekly News' kernel section, but it's a 
> relatively short
> > > > > summary. It recommends other resources such as 
> "Kernel Traffic", but
> > do
> > > > > they beat reading the actual mailing list? Also, are 
> there are other
> > > > > mailing lists of significance for kernel developers?
> > > >
> > > > they are readable - they work as commentary to the 
> actual threads, and
> > its
> > > > quicqer to read. if a specific thread interests you 
> more - go into it.
> > > >
> > >
> > > Cool. That sounds useful.
> > >
> > > > but its more likely that for such a specifically 
> targeted project,
> > reading
> > > > archives of such maling lists, and googling, and 
> reading the kernel's
> > > > source code, and purchasing a copy of alessandro 
> rubini's "linux device
> > > > drivers" book to learn how the kernel internally works 
> - would be more
> > > > productive.
> > > >
> > >
> > > My supervisor recommended "The Linux Kernel Internals", but we
> > > probably could get him to purchase another book. Well, 
> STFEing is the key
> > > to success.
> > >
> > > Regards,
> > >
> > > Shlomi Fish
> > >
> > > > --
> > > > guy
> > > >
> > > > "For world domination - press 1,
> > > >  or dial 0, and please hold, for the creator." -- nob o. dy
> > > >
> > >
> > >
> > >
> > > 
> ----------------------------------------------------------------------
> > > Shlomi Fish        shlomif@t2.technion.ac.il
> > > Home Page:         http://t2.technion.ac.il/~shlomif/
> > > Home E-mail:       shlomif@techie.com
> > >
> > > A more experienced programmer does not make less bugs. He 
> just realizes
> > > what went wrong more quickly.
> > >
> > >
> > > =================================================================
> > > To unsubscribe, send mail to linux-il-request@linux.org.il with
> > > the word "unsubscribe" in the message body, e.g., run the command
> > > echo unsubscribe | mail linux-il-request@linux.org.il
> > >
> > >
> >
> 
> 
> 
> ----------------------------------------------------------------------
> Shlomi Fish        shlomif@t2.technion.ac.il
> Home Page:         http://t2.technion.ac.il/~shlomif/
> Home E-mail:       shlomif@techie.com
> 
> A more experienced programmer does not make less bugs. He 
> just realizes
> what went wrong more quickly.
> 
> 
> =================================================================
> To unsubscribe, send mail to linux-il-request@linux.org.il with
> the word "unsubscribe" in the message body, e.g., run the command
> echo unsubscribe | mail linux-il-request@linux.org.il
> 

=================================================================
To unsubscribe, send mail to linux-il-request@linux.org.il with
the word "unsubscribe" in the message body, e.g., run the command
echo unsubscribe | mail linux-il-request@linux.org.il