[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: Porting and Extending a TCP/IP noise simulator from kernel 2.2.x to 2.4.x



On Thu, 19 Jul 2001, Omer Musaev wrote:

>
>
> > -----Original Message-----
> > From: Shlomi Fish [mailto:shlomif@techst02.technion.ac.il]
> > Sent: Thursday, July 19, 2001 10:41 AM
> > To: Shahar Dag
> > Cc: linux-il@linux.org.il
> > Subject: Re: Porting and Extending a TCP/IP noise simulator
> > from kernel
> > 2.2.x to 2.4.x
> >
> >
> > On Thu, 19 Jul 2001, Shahar Dag wrote:
> >
> > > Hi
> > >
> > > I know that Linux is not my experty, so maybe my question is silly.
> > >
> > > >From software engineering point of view creating non
> > official kernel version
> > > ==> whenever the kernel updated you loose your work. So it
> > seems as the
> > > wrong method.
> > > Don't it be better to write a TCP/IP demon/driver and by
> > this be (almost)
> > > independent at the kernel version
> > >
> >
> > FOA, this daemon has to communicate with the kernel somehow, becaue in
> > Linux it is the kernel that manages the TCP/IP stack.
> > Secondly, putting
> > the arbitration in a module will create a lot of overhead and
> > processing
> > delays, which is something we usually want to avoid if we want to
> > simulate noise in high load networks (e.g: Gigabit Ethernet).
>
> This statement is wrong - after module is initialised, it behaves just like
> the internal code, as far as you do not want to modify the main packet
> dispatcher.
>
> BTW: Are you absolutely sure that you want to simulate noise in GB ethenet?
> Think again:
> card itself should drop all noisy frames, and thus sk_buffs that come from
> giga card are
> marked as though there is no need to checksum those. By applying noise on
> DataLink layer
> you may cause confusion and despair in TCP stack.
>

I think the noise is going to be in the TCP/IP level. But I'll have to ask
my supervisor for details.

Regards,

	Shlomi Fish

> > Regards,
> >
> > 	Shlomi Fish
> >
> > > Shahar
> > > ----- Original Message -----
> > > From: "Shlomi Fish" <shlomif@techst02.technion.ac.il>
> > > To: "guy keren" <choo@actcom.co.il>
> > > Cc: <linux-il@linux.org.il>
> > > Sent: Thursday, July 19, 2001 5:14 AM
> > > Subject: Re: Porting and Extending a TCP/IP noise simulator
> > from kernel
> > > 2.2.x to 2.4.x
> > >
> > >
> > > > On Thu, 19 Jul 2001, guy keren wrote:
> > > >
> > > > >
> > > > > forgive me for this anti-enthusiastic message, but you
> > can implement
> > > your
> > > > > noise simulator without writing a single line of kernel
> > source - you can
> > > > > grab traffic using firewalling rules from kernel mode
> > out into user
> > > mode,
> > > > > and there u can decide whether to let a packet go on,
> > drop it, send it
> > > > > with an error inside, and anything else that a C
> > program allows you to
> > > do.
> > > > >
> > > > > and it'll be able to sustain an ammount of trafic good
> > enough for your
> > > > > needs, i imagine. the question, however, is whether
> > such an approach
> > > > > doesn't defeat the purpose of the lab.
> > > > >
> > > >
> > > > The project is defined to be in the kernel. Otherwise, I
> > think it won't
> > > > be able to scale too much. But maybe we can use the
> > firewalling framework
> > > > in order to better simulate the noise. Hmmm....
> > > >
> > > > > > 1. Is the Networking and TCP/IP stack of kernel 2.4.x
> > very different
> > > > > > internally from that of kernel 2.2.x? From what I
> > understood it was
> > > > > > modified to support multi-threaded processes, but I'm
> > more interested
> > > to
> > > > > > know if thecode remained pretty much the same.
> > > > >
> > > > > some parts were completely rewritten (e.g. the
> > firewalling code). i
> > > think
> > > > > most changes that got to do with what u call
> > 'multi-threaded processes'
> > > is
> > > > > actually not related to threads specifically - but to
> > paralel execution
> > > in
> > > > > general - and got to do with having mroe fine-grained
> > usage of locks
> > > > > inside the kernel, to allow taking better advantage of
> > existance of
> > > > > seveval CPUs in SMP mode.
> > > > >
> > > >
> > > > No, the problem was that in kernel 2.2.x the TCP/IP stack was not
> > > > multi-threaded. That is a blocking accept() call would
> > have blocked the
> > > > entire process, and if one thread sent data the other
> > would get stuck,
> > > > etc. etc.
> > > >
> > > > AlexSh can testify on a case where it became apparent.
> > > >
> > > > > >2. What I thought was to implement some of the noise
> > logic as a patch
> > > to
> > > > > > the kernel, while implementing the arbitrator of it
> > as a separate
> > > module.
> > > > > > During the previous project, the whole thing was
> > implemented as apatch
> > > > > > and they had to reboot constantly.
> > > > > >
> > > > > > My question is: can the logic that decides whether to
> > drop, ruin, or
> > > how
> > > > > > much time to delay a packet, can be implemented as a
> > separate module
> > > that
> > > > > > will communicate with the main kernel somehow?
> > > > >
> > > > > yes. you could write your own 'firewall' module, and
> > plug it into the
> > > > > kernel. the internel API in kernel 2.4 supports
> > plugging of several API
> > > > > modules, chained on-top of each other. this way, you
> > don't need to
> > > modify
> > > > > the kernel's code at all, and write everything as a module.
> > > > >
> > > >
> > > > If you say so - it might save us a lot of work and we may
> > not need to use
> > > > the original code at all (except as a refernce).
> > > >
> > > > > > 3. I'd like to use kernel debuggers like SGI's kdb
> > and Personal ICE
> > > > > > (http://pice.sourceforge.net/) to aid in analysing
> > errors. I know they
> > > are
> > > > > > not distributed with the kernel of Linus, but I
> > suppose they can be
> > > > > > applied quite easily.
> > > > > >
> > > > > > Assuming the main kernel (which has both or one of
> > the debuggers
> > > compiled
> > > > > > into it) has an OOPS fault, will they allow me to diagnose it?
> > > > >
> > > > > you don't need a debugger in order to get an OOPS
> > message diagnosed.
> > > klogd
> > > > > does that for you, if you maintain an up-to-date symbol
> > map. it'll
> > > > > translate the stack into kernel symbols offsets, and
> > you could locate
> > > the
> > > > > relevant instruction by dissassembling your module.
> > > > >
> > > >
> > > > Yes, but kernel debuggers enable me to see the value of
> > my variables. (!)
> > > > and they are interactive and I can do more at that point
> > than just reboot
> > > > a safe kernel and look at my source code. At least that's
> > how it is in
> > > > Soft-ICE, which Personal ICE aims to resemble.
> > > >
> > > > > > 4. Naturally, to test the module, I'll need to
> > simulate network
> > > activity. I
> > > > > > can do that with actual packets that arrive from
> > different machine (or
> > > > > > from two different Ethernet cards on the same
> > computer - %-)) but I
> > > could
> > > > > > also build an internal virtual network, and do some
> > manipulations
> > > there.
> > > > > >
> > > > > > Or could I? I think depending on packets that arrive
> > from the hardware
> > > at
> > > > > > the early stages of the project is a bad idea<tm>,
> > because kernel
> > > > > > programming is bad enough without messing with the hardware.
> > > Naturally, we
> > > > > > will make sure it works with various hardware, but
> > can we postpone it
> > > to
> > > > > > later stages? Should we do that?
> > > > >
> > > > > you could write your code so that you could compile it
> > both in user mode
> > > > > and in kernel mode. then you could write the logics in
> > user mode and
> > > test
> > > > > it there, and only when its debugged - put it in the
> > kernel. i could
> > > send
> > > > > you the syscall tracker module's sources as an example
> > of one way this
> > > > > could be done.
> > > > >
> > > >
> > > > Sounds good. If you can, please put the sys-call tracker
> > source on the web
> > > > so my partner and my project's supervisor will be able to
> > look at it.
> > > >
> > > > I think gdb still beats Personal ICE hands-on, but the
> > kernel debuggers
> > > > will probably prove to be useful too.
> > > >
> > > > > > 5. Simulating network noise may be of interst to Comp. Net.
> > > researchers
> > > > > > and Net-Admins world-wide. For instance: can MOSIX or
> > Beowulf work
> > > with
> > > > > > this patch installed on one or more of the clusers'
> > computers? I'd
> > > like to
> > > > > > find out.
> > > > > >
> > > > > > So, what do you recommend regarding the issue of
> > whether or not it
> > > should
> > > > > > be integrated into the main kernel? I think the patch
> > to the kernel
> > > should
> > > > > > be #ifdef'd because delaying, ruining, or dropping packets
> > > deliberately is
> > > > > > something most people don't want to enable.
> > > > > >
> > > > > > Now, Linus objected to the MOSIX code be integrated
> > into the kernel
> > > that
> > > > > > way, so I don't know if he'll accept such patch. I
> > guess I'll have to
> > > ask
> > > > > > him. Of course, the patch can be maintained vis-a-vis
> > with the kernel
> > > > > > source (like MOSIX), but it would be a better idea if
> > it is in the
> > > Linus'
> > > > > > source tree.
> > > > >
> > > > > since it can be done in user mode, or as a kernel module without
> > > modifying
> > > > > any line of code in the kernel, there's no point in
> > integrating this
> > > into
> > > > > the kernel's sources.
> > > > >
> > > >
> > > > If we indeed end up with a user-level/kernel module
> > arbitrator, then we
> > > > can indeed distribute and maintain it separately from the
> > kernel. However,
> > > > if many researchers and Internet workers find it useful,
> > Linus may decide
> > > > to include it there.
> > > >
> > > > > > 6.
> > > > > > Do you recommend joining or monitoring the
> > linux-kernel mailing list.
> > > I
> > > > > > know the amount of daily messages there is very big,
> > much more than
> > > all
> > > > > > the other mailing-list I am a member of. I could
> > request a bigger
> > > quota to
> > > > > > accomodate for it, but I'll probably press the "D"
> > button maniacly.
> > > > >
> > > > > i think i see muli waving his hand enthusiastically
> > with a 'yes' vote :)
> > > > >
> > > >
> > > > I could put it in a separate directory. Put the messages
> > of every week (or
> > > > every day?) in a separate mailbox, and maybe get a
> > separate account.
> > > >
> > > > > > There are various web resources that summarize the
> > activity there. I
> > > read
> > > > > > the Linux Weekly News' kernel section, but it's a
> > relatively short
> > > > > > summary. It recommends other resources such as
> > "Kernel Traffic", but
> > > do
> > > > > > they beat reading the actual mailing list? Also, are
> > there are other
> > > > > > mailing lists of significance for kernel developers?
> > > > >
> > > > > they are readable - they work as commentary to the
> > actual threads, and
> > > its
> > > > > quicqer to read. if a specific thread interests you
> > more - go into it.
> > > > >
> > > >
> > > > Cool. That sounds useful.
> > > >
> > > > > but its more likely that for such a specifically
> > targeted project,
> > > reading
> > > > > archives of such maling lists, and googling, and
> > reading the kernel's
> > > > > source code, and purchasing a copy of alessandro
> > rubini's "linux device
> > > > > drivers" book to learn how the kernel internally works
> > - would be more
> > > > > productive.
> > > > >
> > > >
> > > > My supervisor recommended "The Linux Kernel Internals", but we
> > > > probably could get him to purchase another book. Well,
> > STFEing is the key
> > > > to success.
> > > >
> > > > Regards,
> > > >
> > > > Shlomi Fish
> > > >
> > > > > --
> > > > > guy
> > > > >
> > > > > "For world domination - press 1,
> > > > >  or dial 0, and please hold, for the creator." -- nob o. dy
> > > > >
> > > >
> > > >
> > > >
> > > >
> > ----------------------------------------------------------------------
> > > > Shlomi Fish        shlomif@t2.technion.ac.il
> > > > Home Page:         http://t2.technion.ac.il/~shlomif/
> > > > Home E-mail:       shlomif@techie.com
> > > >
> > > > A more experienced programmer does not make less bugs. He
> > just realizes
> > > > what went wrong more quickly.
> > > >
> > > >
> > > > =================================================================
> > > > To unsubscribe, send mail to linux-il-request@linux.org.il with
> > > > the word "unsubscribe" in the message body, e.g., run the command
> > > > echo unsubscribe | mail linux-il-request@linux.org.il
> > > >
> > > >
> > >
> >
> >
> >
> > ----------------------------------------------------------------------
> > Shlomi Fish        shlomif@t2.technion.ac.il
> > Home Page:         http://t2.technion.ac.il/~shlomif/
> > Home E-mail:       shlomif@techie.com
> >
> > A more experienced programmer does not make less bugs. He
> > just realizes
> > what went wrong more quickly.
> >
> >
> > =================================================================
> > To unsubscribe, send mail to linux-il-request@linux.org.il with
> > the word "unsubscribe" in the message body, e.g., run the command
> > echo unsubscribe | mail linux-il-request@linux.org.il
> >
>
> =================================================================
> To unsubscribe, send mail to linux-il-request@linux.org.il with
> the word "unsubscribe" in the message body, e.g., run the command
> echo unsubscribe | mail linux-il-request@linux.org.il
>



----------------------------------------------------------------------
Shlomi Fish        shlomif@t2.technion.ac.il
Home Page:         http://t2.technion.ac.il/~shlomif/
Home E-mail:       shlomif@techie.com

A more experienced programmer does not make less bugs. He just realizes
what went wrong more quickly.


=================================================================
To unsubscribe, send mail to linux-il-request@linux.org.il with
the word "unsubscribe" in the message body, e.g., run the command
echo unsubscribe | mail linux-il-request@linux.org.il