*Internet2 QoS Working Group Meeting* January 29, 2002 Tempe, Arizona *Attendees* Ben Teitelbaum (chair), Internet2 Terry Gray, Washington Alistair Munro, U4EA Chuck Wolf, Georgetown Clark Gaylord, Virginia Tech Brian Kaye, New Brunswick Sia Maher, Cisco Tricha Anjali, Georgia Tech Jaudelice Beckmann, Georgia Tech Mike Contino, Penn State Kurt Jeschke, Penn State Shumon Huque, Penn Ben Chinowsky (scribe), Internet2 *Discussion* BT: all official WG biz is on mailing list, but it's nice to talk. main agenda item is followup & continuation of discussion about implications of QoS appliances. but first, an update on QoS WG activities. ...introductions... BT: note that we follow the IETF open WG model, so you're all welcome to consider yourselves members if you like. Review of current work -- we have several design teams: - Premium work wrap-up (...outlines Premium...) we learned you can build it, but it's very hard to convince anyone to deploy it in their network. not dead but being suspended. GEANT has an ambitious Premium project, led by Mauro Campanello. Talk to me if you're interested in the Premium "post mortem" work; a rough draft exists. - Scavenger Service. about a year old. (...outlines "bottom-feeding service"...) motivation is partly ease, partly that "there are self-policing users out there", like high-energy physicists who send their huge data sets at night -- Scavenger is like sending at night 24 hours a day, thus actually makes it possible to get your data sent quicker. CDN networks, backups -- even cases where you're competing against yourself. scavenger is out there and gaining traction, in particular dorm subnets being marked this way, SLAC and CERN are experimenting with it, up to about 1% of Abilene traffic...GRAPE project (Konishi)...John Crowcroft and others pushing in context of Euro Data Grid...way more modest than Premium, but deploying way more quickly. Gaylord: any measurements of positive effect? BT: in the lab yes, in the field no. Kaye: right, 1% of traffic on Abilene isn't enough to "rise above the noise" Gaylord: if there was very heavy congestion, but little Scavenger use, a dedicated 1% link share could actually result in Scavenger getting *better* service BT: right, under pathological conditions Scavenger can actually get better service, but it's not an economically stable situation Huque: how many schools are scavenging dorms? BT: 5-10. top scavenging ASes a few weeks back were: RIT, MOREnet, UMD, Moscow State University, and VA-Tech Huque: see: www.itec.oarnet.abilene.netflow for the data BT: 001000 is Scavenger DSCP Gaylord: that's our academic traffic -- not supposed to be Scavenger -- don't get treated that way; guess we should change our codepoint! BT: contra IETF norms, we used a well-known codepoint BT: current WG activities (cont'd) - ABE-like service. discussed but no team yet assigned. idea is to split BE into two classes that trade off loss and delay; "different but equal". TEG: TCP wouldn't want to be in low-delay/high-loss class BT: right. ABE inventors have an elaborate queueing discipline to preserve a formal and strict notion of fairness (green cannot hurt blue) (see www.abeservice.com); it's not clear that you need that discipline to get something useful; a similar effort called BEDS came out of Nortel labs; it would be good to have a positive story about helping delay-sensitive apps. BT: current WG activities (cont'd) - apps QoS needs design team. 65 applicants for $10K fellowship. grad student at U. College London selected, Dimitrios Miras, I'll meet him Friday. Objective is to write a survey paper that "really drills down for a few key applications" -- write in such a way that it bridges gap between apps developers and net designers. want to dispel myths about QoS that apps need -- show how apps can be "more internet-friendly" and disseminate best-practices "hooks" to help applications adapt successfully (e.g. MIT congestion manager, TCP MIB, RTP) BT: so, should we add a fifth design team for QoS appliances? Where do we go from here? Alistair, perhaps you could bridge us over into a discussion of QoS appliances by talk a little about your product? AM: started on multi-use Euro packet data networks...looking at growth of traffic led us to algorithms for trading loss & delay vs. throughput. What we claim to do better is provide more predictable service, under conditions where use of other algorithms lead to breakdown and uncertainty. Packeteer boxes seem to have a role at server end when you're trying to back off incoming TCP connections TEG: What are your definitions of "server" and "source"? AM: server is e.g. database receiving many requests, source is client. most other companies concentrate on bandwidth management -- most of 25 or so surveyed by Craig at Missouri. we're interested in bandwidth for mobile communications though -- not relevant much to Internet2 now, but possibly in longer term. we only launched in December, no real customers yet, so no real-world verification that our product works. to sum up, we're very interested in finding out the fate of these boxes. we don't necessarily believe in QoS appliances -- our current "bubble on the line" approach doesn't scale to thousands of router ports, must be integrated into routers eventually. TEG: so you want your stuff built into routers? AM: we're talking to residential gateway people TEG: user interface? AM: none TEG: so someone else decides what user gets? AM: yes. a signaling app would do that configuration for you. what I really don't want to see is the construction of very complex signaling systems, rebuilding POTS etc. TEG: the trend does seem to be in that direction AM: my opinion is that the telcos are trying to preserve their current biz model while gaining the cost-effectiveness of IP...this group doesn't seem to have the right constituency, e.g. I see no video people here. ISPs have a bandwidth glut, so they don't want our stuff. TEG: same with multicast; service providers trying to charge for what is in essence a bandwidth reducing technology AM: more successful in enterprise than with ISPs -- a single service that they want to differentiate BT: there's an argument that says that if you assume that demand continues to grow exponentially and you look at these devices as providing you a fixed efficiency coefficient; then you delay having to upgrade your circuit only once, after which you are right back on an exponential demand curve. Kaye: there's a feedback loop such that you'll also likely lower the slope of the increase, in addition to your one-time gain. Contino: if it's one time, Alistair is out of business. Gaylord: so, ultimately you can't stop the rate at which offered traffic grows? BT: that was my assumption Gaylord: I'm not sure that's so in a consumer-driven model -- let congestion "do good things for the service" -- still exponential, but with a different coefficient. But just one service provider can't do this. bandwidth management devices let you do this the way you want instead of just throwing more bandwidth at it. AM: feedback loop, yes, with various levers such as pricing. Wolf: at Georgetown we don't charge for data, can only throw so much money at the problem; when we explain the situation to students we don't get a lot of pushback -- that explanation is part of the feedback too. BT: tragedy of the commons. I'm struck by how much campus network ops resist charge-back schemes Wolf: easier to get away with restrictions at a cheap school...revenue streams from phone system have dried up TEG: I'm worried we'll get the worst of both the billing and the middlebox models. Internet is looking more like the phone system, with ever-growing focus on counting things. billing by data per source is more workable than an approach that assumes you can track every source-destination pair. like a pusher, you don't charge much to start. also we needed to subsidize to head off balkanization of the net resulting from people saying "we can do it cheaper than you", then going off and doing so. we're setting up a two-tier service, having our first conversation with the dorms people. in 12 months we've gone from producing half as much as we consume, to the reverse. if we constrain bandwidth, no one's happy: P2Pers will try anyway, other users will say "I'm a good guy, track me individually." heading toward counting packets per user, I don't like it one bit. Kaye: that's what we do. everyone on campus has static IP and the following quotas: 200 MB bidirectional summed in prime time, 800 MB otherwise, 30 GB monthly limit -- has cut our performance complaints dramatically. exceed and you get a bill. there are occasional mistakes -- one student ran up a $1K bill on a weekend. call them in, talk, don't always have to pay the whole thing. web site shows your usage up to 15 minutes ago. this is all accounting. on the enforcement side, we shut the really bad people down entirely. BT: would you consider exempting Scavenger from this regime? Kaye: hadn't thought about it. BT: ultimately we'd like to have Scavenger markings set by Kazaa etc. Contino: billing cycle? Kaye: monthly, with email notification when approaching daily and monthly limits. each user is in a single big VLAN for all residences; there are other VLANs for other uses -- 6 or 7 total. if they try to pick their own IP address, it doesn't go anywhere. Contino: they could spoof MAC address Kaye: yes, but that attack is harder. You need to know a valid MAC address that is already entrolled. The registration process does pick up the MAC address from the card in most cases unless you are registering from a different machine. TEG: when does a registered MAC address expire? Kaye: The RESNET VLAN is flushed on May 1 or whatever is the offical last day students can be in residence for the term. IF they stay they must re-register. Anjali: how much allocated? Kaye: 1/5 of 30 Mbps (i.e. about 6Mbps), which is for about 1200 students in wired residences. We have about a 70% rate of students in residence with computers. No QoS appliances currently, but we're looking at Packeteer TEG: they seem to think their classification capability is a strength, others prefer the "Whack-A-Mole model"...Alistair, where are you? AM: most customers want to be able to classify applications, e.g. PBX operators want to give priority to voice. not hard to track by well-known protocols. TEG: so will we have nightly signature updates for the latest rogue apps? (...seems like a possibility...) Kaye: do you pay separately for inbound and outbound? our service is asymmetric Gaylord: we have symmetric, 45 Mbps both ways...we pay for an OC3 which we can use anywhere in the state...could bankrupt the country this way... BT: it's getting late, so, do we want a QoS appliance WG? Gaylord: yeah, but look at other loops also -- broader than just QoS appliances BT: right, look at from many perspectives, P2P, middleboxes... AM: I agree, should be broader focus than middleboxes Kaye: (more on Packeteer) AM: all middleboxes can stop DoS attacks TEG: right, all can throttle TCP BT: another angle is E2Epi -- these boxes are slow, generally an order of magnitude slower than the network; the point of Stas's math-heavy slide was that middleboxes increase the probability of small amounts of random packet loss, which a) are hard to detect and b) can really kill performance TEG: we have apps where we prefer no service to degraded service, as long as user knows what's happening so they don't think the network is just broken...Ben, how about trying to collect experiences re how people have tried to solve the bandwidth management problem? BT: yes, today's New Brunswick discussion is a good example Gaylord: we (Network Virginia) found that having a forum on bandwidth management solutions was very popular. TEG: Educom published a collected-wisdom book on net strategies a few years ago. Kaye: we also don't allow anyone on campus to run a server of any kind. finding them is not a big priority, but every so often we look for spikes and scan the ports of the offenders. (...wherever you plug in on campus you're still on your machine's own VLAN...) we don't go after offenders who don't use a lot of bandwidth. Ben C.: what happened to the economics working group? BT: they were academics -- I think that when they realized how hard it is to experiment with pricing models on the backbone, they lost interest. burned brightly if briefly though -- there sure was a lot of interest at that DC BoF. BT: I know that Joe St. Sauver is also interested in pursuing this; let's throw Terry's experience-collection-as-step-toward-BPs idea, at least via circulating Ben C.'s notes, which contain it, and see what traction we get [sense of the meeting: do something, yes] FIN