misc-mirrors/article_scraper
1
0
Fork 0
mirror of https://gitlab.com/news-flash/article_scraper.git synced 2025-07-07 16:15:32 +02:00
Code Releases Activity

add two new tests

Browse source
This commit is contained in:
Jan Lukas Gernert 2023-02-28 18:28:55 +01:00
parent a1c07d436f
commit 451dd61547
5 changed files with 2033 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

311
resources/tests/readability/archive-of-our-own/expected.html Normal file
View file

@ -0,0 +1,311 @@
<article><div role="article" id="readability-page-1">
<h3 id="work">
Chapter Text
</h3>
<p>
Izuku was struggling to understand how he had even managed to get here, seated before the archvillain of Japan with only a sense of dread to keep him company. All Might sat concealed in an observation room, of the firm opinion that he could only aggravate the prisoner and he sent Izuku off with a strained smile. A vague haze hovered over Izuku’s memory. It started with a simple conversation gone astray on a long drive home.
</p>
<p>
“So, who is All For One? Do we know anything about him beyond what you told me before? He’s been imprisoned for months now.” Izuku remembered asking All Might from the backseat of the car as Detective Tsukauchi leisurely drove along a sprawling highway.
</p>
<p>
Playing on the car radio was an aftermath report of a villain attack in downtown Tokyo. Izuku caught the phrase “liquid body” from the female reporter before Detective Tsukauchi changed the channel.
</p>
<p>
“Nope. Still nothing. No one really wants to speak to him,” All Might had replied brightly. “He gives off polite airs, but he’s a piece of work.” All Might’s mostly obstructed shoulders in the front seat shrugged. “Not much you can do with someone like him. Everything that comes out is a threat or taunt.” All Might carefully waved his hand in a circular motion towards the side of his head.
</p>
<p>
“No one’s even made it through a full interview with him, from what I’ve heard,” Detective Tsukauchi added from behind the wheel. “He plays mind games with them. The prison also has a “no recent events” policy on any discussions with him as well. Just in case he ends up with ideas or has some means of communicating. Given that people only want to ask him about current events, it doesn’t leave much to talk about.”
</p>
<p>
“Wait, they still don’t know what Quirks he has?” Izuku asked exasperatedly. “They can’t if there’s still an information block on visits.”
</p>
<p>
“Nope. We have no idea what he can do. They can run DNA tests, but it’s not like anyone apart from him even knows how his Quirk works. They could get matches with any number of people, but if they’re not in a database then we can’t cross-reference them anyway. Even if they run an analysis, the data doesn’t mean anything without the ability to interpret it,” All Might gestured with a skeletal finger. “It’s a waste of time after the initial tests were conducted. They weren’t game to MRI him either, given he’s definitely got a Quirk that creates metal components.”
</p>
<p>
“No one’s bothered to ask him anything about… anything?” Izuku asked, dumbfounded. “He must be around two-hundred years old and people can’t think of a single non-current affairs thing to ask him?”
</p>
<p>
In some ways it was unfathomable that they’d let a potential resource go to waste. On the other hand, said potential resource had blown up a city, murdered numerous people and terrorised Japan for over a century. At the very least.
</p>
<p>
“Well, I tried to ask him about Shigaraki, but he didn’t say much of anything really. Some garbage about you being too dependent on me and him letting Shigaraki run wild and how he just wanted to be the ultimate evil,” All Might shrugged again. “He spends too much time talking about nothing.”
</p>
<p>
Izuku shifted his head onto his arm. “But, that’s not really nothing, is it?”
</p>
<p>
“What do you mean?” Izuku had the feeling that All Might would have been looking at him with the <i>you’re about to do something stupid aren’t you</i> expression that was thankfully becoming less common.
</p>
<p>
“Well, he clearly doesn’t know anything about us, All Might, if he thinks that you’re just going to let go of me after not even two years of being taught. Maybe Shigaraki was dependent on adult figures, but I don’t even remember my dad and mum’s been busy working and keeping the house together. I’ve never had a lot of adult supervision before,” Izuku laughed nervously. “I had to find ways to keep myself entertained. If anything, I’m on the disobedient side of the scale.” All Might outright giggled.
</p>
<p>
“I’ll say, especially after what happened with Overhaul. I’m surprised your mother let you leave the dorms again after that.”
</p>
<p>
“I’m surprised she didn’t withdraw and ground me until I was thirty.”
</p>
<p>
“Oh? That strict?” Tsukauchi asked.
</p>
<p>
“She has her moments,” Izuku smiled fondly. “Do you think she’d agree to me asking the archvillain of Japan about his Quirk?” Izuku asked, only partially joking. There was an itch at the back of his head, a feeling of something missing that poked and prodded at his senses.
</p>
<p>
All Might coughed and sprayed the dash with a fine red mist. “Absolutely not! I forbid it!”
</p>
<p>
“That’s exactly why I’m asking her and not you,” Izuku grinned from the backseat.
</p>
<p>
“He’s evil!”
</p>
<p>
“He’s ancient. You honestly don’t wonder about the sort of things someone with that life experience and Quirk would have run across to end up the way he did?”
</p>
<p>
“Nope, he made it perfectly clear that he always wanted to be the supreme evil,” All Might snipped through folded arms.
</p>
<p>
“Yeah, and I’ll just take his word for that, won’t I?” Izuku grinned. “If he does nothing but lie, then that’s probably one too, but there’s a grain of truth in there somewhere.”
</p>
<p>
“What would you even do? Harass him into telling you his life story?” All Might sighed.
</p>
<p>
“Not when I can kill him with kindness. Who knows, it might even be poisonous for him.”
</p>
<p>
“You’re explaining this to your mother. Teacher or not, I’m not being on the receiving end of this one.”
</p>
<p>
Izuku blinked for a moment. “You’ll let me?”
</p>
<p>
“I’m not entirely for it, but any prospective information on what influenced Shigaraki can only be a good thing. If anything goes south we can pull you out pretty easily. Just be aware of who and what you’re dealing with.” Struggling, All Might turned a serious look to Izuku around the side of the seat. “<i>Only</i> if your mother gives the okay.”
</p>
<p>
The conversation turned to school for the rest of the way.
</p>
<p>
It might have been curiosity or it might have been the nagging sensation that chewed at his brain for the three weeks that he researched the subject of the conversation. All For One was a cryptid. Mystical in more ways than one, he was only a rumour on a network that was two-hundred years old. There were whispers of a shadowy figure who once ruled Japan, intermingled with a string of conspiracies and fragmented events.
</p>
<p>
Izuku had even braved the dark web, poking and prodding at some of the seedier elements of the world wide web. The internet had rumours, but the dark web had stories.<br></p>
<p>
An implied yakuza wrote about his grandfather who lost a fire manipulation Quirk and his sanity without any reason. His grandfather had been institutionalised, crying and repeating “he took it, he took it” until his dying days. No one could console him.
</p>
<p>
Another user spoke of a nursing home where a room full of dementia residents inexplicably became docile and no longer used their Quirks on the increasingly disturbed staff. The nursing home erupted into flames just before a court case against them commenced.
</p>
<p>
A user with neon pink text spoke of how their great-great-great-great grandmother with a longevity Quirk had simply aged rapidly one day and passed away in her sleep, her face a mask of terror. No cause had ever been found.
</p>
<p>
A hacker provided a grainy CCTV recording of a heist and a scanned collection of documents from over a century ago, where there was a flash of light and entire bank vault had been emptied. What separated it from the usual robbery was that it contained a list containing confidential information on the Quirks of the First Generation. Izuku had greedily snavelled up and saved the video and documents to an external hard drive.
</p>
<p>
Paging through, Izuku saw someone recount how their Quirkless uncle had developed a warp Quirk and gone from rags to riches under a mysterious benefactor. A decade ago, the uncle had simply disappeared.
</p>
<p>
Numerous and terrifying, the stories were scattered nuggets of gold hidden across the web. They’d never last long, vanishing within hours of posting. Izuku bounced from proxy to proxy, fleeing from a series of deletions that seemed to follow Izuku’s aliased postings across snitch.ru, rabbit.az, aconspiracy.xfiles and their compatriots.
</p>
<p>
After thirty-two identity changes (all carefully logged in a separate notebook), a large amount of feigning communal interest in a lucky tabloid article on All For One which had been released at the start of the first of the three weeks, Izuku hung up his tinfoil hat and called it a month. He haphazardly tossed a bulging notebook into his bookshelf and lodged his hard drive in a gap containing seven others and went to dinner.
</p>
<p>
It took another week to present his research to All Might and Tsukauchi, whose jaws reached the proverbial floor.
</p>
<p>
“We never found any of this,” the Detective Tsukauchi exclaimed. “How did you find all of it?”
</p>
<p>
“I asked the right people. Turns out criminals have very long and very unforgiving memories,” Izuku explained through sunken eyes. “There’s more than this that could be linked to him, but these ones seem to be the most obvious.”
</p>
<p>
“They would do, you can’t be head of the underworld without making an army of enemies,” All Might agreed. “You know, if you can get any more information about these events, I think you’ll give people a lot of peace of mind.”
</p>
<p>
“Provided mum agrees to it.”
</p>
<p>
“Only if she agrees to it.”
</p>
<p>
It took another month to convince his mother, who eventually gave in once All Might provided an extremely comprehensive schedule of how the visitations and any resulting research would be carefully balanced against Izuku’s schoolwork and internship.
</p>
<p>
The day of the visit finally arrived, four months after the initial conversation, much to Izuku’s dismay.
</p>
<p>
Izuku remembered how he had arrived, with the Detective and All Might escorting him through its sterile, white innards. A list of rules rattled off at the gate, “no current affairs” was chief among them and an assertion that he’d be dragged from the room if need be if Izuku was to breach any of them. No smuggling of communication devices, no weapons, no Quirks, nothing that could compromise the prisoner’s secure status.
</p>
<p>
Heavily armoured and drilled guards leading him underground into the deepest bowels of the Tartarus complex.
</p>
<p>
Izuku understood the rules, dressed casually in a cotton t-shirt with “Shirt” printed across it in haphazard English and clutching at a carefully screened and utterly blank notebook.
</p>
<p>
Across from him, behind reinforced glass, the archvillain of Japan was bound and unmoving.
</p>
<p>
“Hello,” Izuku initiated uncertainly. His skin had been crawling the moment he crossed the threshold, a memory of the encounter and escape at the Kamino Ward months ago.
</p>
<p>
“Ah, All Might’s disciple,” drawled All For One, “is he too cowardly to come himself? Yet I don’t hear the garments of a hero.” With hardly a word out, All For One had already lunged for the figurative jugular.
</p>
<p>
A stray thought of <i>how does he know who I am if he’s blind and isn’t familiar with me?</i> whispered its way through Izuku’s head.
</p>
<p>
“Oh, no,” Izuku corrected hastily, almost relieved at the lack of any pretence, “I asked if I could talk to you. This isn’t exactly hero related.”
</p>
<p>
“I’m surprised he said yes.” While there was little by way of expression, Izuku could just about sense the contempt dripping from the prisoner’s tone. It wasn’t anything he wasn’t expecting. Kacchan had already said worse to him in earlier years. Water off a duck’s back.
</p>
<p>
“Well, he’s not my legal guardian, so I think you should be more surprised that mum said yes. She’s stricter with these things than All Might,” Izuku corrected again. “Mum gave the okay, but that was a stressful discussion.” And there it was, a miniscule twitch from the man opposite. A spasm more than anything else. <i>Interesting.</i> Pinned down as he was, the prisoner oozed irritation.
</p>
<p>
“At least your mother is a wise person. I wonder why the student doesn’t heed all of the advice of the teacher.” All For One’s tone didn’t indicate a question, so much as an implicit statement that All Might wasn’t worth listening to in any capacity. Kacchan would have hated the comparison, but the hostility had an almost comfortable familiarity. “He no doubt warned you off speaking to me, overprotective as he is, but here you are.”
</p>
<p>
Izuku found himself smiling at the thought of Kacchan’s outrage if he ever found out about the mental comparison as he replied. “I don’t think it’s normal for anyone my age to listen completely to their teachers. We pick and choose and run with what works best for us. He warned me, but I’m still here. Mum warned me as well, but I think she cared more about the time management aspect of it."
</p>
<p>
“Is that a recent development?” All For One probed.
</p>
<p>
“Not really. My old homeroom teacher told me not to bother applying to U.A.” His mother’s beaming face had carried Izuku through the cheerful and resolute signing of that application form.
</p>
<p>
“I see you followed their advice to the letter,” came the snide, dismissive reply.
</p>
<p>
Izuku hoisted up his legs and sat cross-legged in his seat. Leaning slightly forward as he did so as to better prop up his notebook.
</p>
<p>
“You’re a walking contrarian, aren’t you? All Might told me about his run ins with you. What someone does or doesn’t do really doesn’t matter to you, you’ll just find a way to rationalise it as a negative and go on the attack anyway. What you’re currently doing is drawing attention away from yourself and focusing it on me so you can withhold information.” Izuku flipped open his notebook and put pen to paper. “You’ve got something fairly big to hide and you diverting attention exposes that motivation as existing anyway. The only real questions here are what and why?” Izuku paused in mortification as the man opposite’s lips parted. “I just said that aloud, didn’t I?”
</p>
<p>
Of the responses Izuku had expected, it wasn’t laughter. Unrestrained, Izuku would have expected a violent outburst. In this situation, he would have expected another scathing comment. Instead, All For One laughed breathily, leaning into his bonds. Wheezingly he spoke, “I’ll have to change tactics, if that one’s too transparent for you. How refreshing.”
</p>
<p>
Doing his best not to glow a blinding red and simultaneously pale at the interest, Izuku carried on. “I add it to the list when you do. I’m not emotionally involved enough to really be impacted by what you’re saying. I know about you in theory, but that’s it. Maybe All Might has a history with you, but I don’t really know enough about you personally to…”
</p>
<p>
“Care,” All For One supplied, somewhat subdued as he struggled to breathe. “You’re only here to satisfy your curiosity as to whether or not the stories were true.”
</p>
<p>
Izuku nodded, scratching at his notebook with his left hand. “Yes and no, I’m actually here to ask you about how your Quirk works.” <i>For now.</i></p>
<p>
Another chortle, more restrained that the last.
</p>
<p>
"What makes you think others haven’t already asked?” Had All For One been unrestrained, Izuku could imagine the stereotypical scene of the villain confidently leaning back in some overblown chair in a secret lair, drink of choice in hand, if the tone of voice was any indication. Deflections aside, the man easily rose to each comment.
</p>
<p>
“Whether or not they asked it’s irrelevant if they can’t read the answers.” Answers didn’t matter if the people involved were too attached to read into the answers. If none of the interviewers had managed a full interview, then it seemed unlikely that any sort of effort was put into understanding the villain.
</p>
<p>
“And you think you can? What expertise do you hold above theirs?” Doubt and reprimand weighted the words. Oddly enough, had Izuku been any younger he could have mistaken the man for a disapproving parent rebuking an overly ambitious child. Albeit an extremely evil one.
</p>
<p>
Izuku inhaled shortly and went for it. “If there’s something I know, it’s Quirks and how they work. Maybe I don’t know you, but I don’t really need to. Quirks fall under broad categories of function. You can take and give, consent doesn’t seem to be a factor. You either can’t “see” certain types of Quirks or you need to have prior knowledge of it before you take it with what I know about your brother. Despite your <i>nom de guerre</i>, because we both know it’s not your real name, you have a history of giving multiple Quirks and causing brain damage to the receiver. You clearly aren’t impacted by those same restrictions, so it must either alter your brain mapping or adjust functions to allow for simultaneous use and storage. It also must isolate or categories the Quirks you stock, because from the few people who do remember you, you creating certain Quirks is always in the context of giving them to someone else meaning there’s probably an inherent immunity to stop it from tainting your own Quirk with a mutation,” Izuku mumbled, almost to himself. “The only thing really in question about your Quirk is the finer details and whether or not you need to maintain those features or if they’re inherent and your hard limit for holding Quirks.”
</p>
<p>
There was silence, for only a moment. “If only my hands were free, I would clap for such a thoughtful assessment. Clearly you’re not all brawn,” All For One positively purred. “Speculate away.” A wide and slightly unhinged smile was directed at Izuku.
</p>
<p>
It was all Izuku could do not to wince at the eagerness. An image of a nervous All Might, hidden in the observation room above with the grim-faced prison staff, came to mind.
</p>
<p>
“I note that you said thoughtful and not correct,” and Izuku breathed and unsteadily jotted it down in his notebook. “You don’t seem bothered by the guess.”
</p>
<p>
“Few people live long enough to question my Quirk, let alone have the talent to guess so thoughtfully at its functions. It seems we share a hobby.” There was something terribly keen in that voice that hadn’t been there before, twisting itself through the compliment.
</p>
<p>
“I suppose it helps that you’re playing along out of boredom,” Izuku verbally dodged, unease uncoiling itself from the back of his mind.
</p>
<p>
“I <i>was</i> playing along out of boredom,” All For One corrected smoothly. “Now, I’m curious. Admittedly, my prior assumptions of you weren’t generous, but I’ve been too hasty in my assessments before.”
</p>
<p>
“I’ll pack up and leave now if that’s the case,” Izuku replied with only half an ear on the conversation as the words on his page began to drastically expand to distract himself from the building anxiety.
</p>
<p>
“Sarcasm, so you do have characteristics of a normal teenager. Your willingness to maim yourself has often left me wondering…”
</p>
<p>
“You’re deflecting again,” Izuku observed. “I’m not sure if that’s a nervous habit for you or if you’re doing it because I’m close to being right about your Quirk. That being said, I don’t think you know what a normal teenager is if Shigaraki is any indication. He’s about seven years too late for his rebellious phase.”
</p>
<p>
“I’m hurt and offended,” came the amused reply.
</p>
<p>
“By how Shigaraki ended up or your parenting? You only have yourself to blame for both of them.”
</p>
<p>
“How harsh. Shigaraki is a product of society that birthed him. I can’t take credit for all of the hard work,” All For One laid out invitingly. Perhaps someone else would have risen to the bait, but Izuku was already packing his mental bags and heading for the door.
</p>
<p>
Clearly the prisoner’s anticipation had registered poorly with someone in the observation room, because a voice rang through the air. “Time’s up Midoriya-kun.”
</p>
<p>
“Okay!” Izuku called back and etched out his last thoughtful of words, untangled his legs and rose to his feet.
</p>
<p>
“What a shame, my visitations are always so short,” All For One spoke mournfully.
</p>
<p>
“Well, you did blow up half a city. They could have just let you suffocate instead. Same time next week, then?” Izuku offered brightly, notebook stuffed into a pocket and was followed out the door by wheezing laughter.
</p>
<p>
It was only after he had made it safely back to the communal room where All Might waited did he allow the spring to fade from his step and discard his nervous smile. Shuddering, he turned to All Might whose face was set in a grimace.
</p>
<p>
“I won’t say I told you so,” All Might offered, perched on the edge of his couch like a misshapen vulture.
</p>
<p>
“He’s… not really what I was expecting. I was expecting someone, more openly evil.” Izuku allowed himself to collapse into the leather of the seat. He shakily reached for the warm tea that had been clearly been prepared the moment Izuku left the cell. “I suppose he does it to lull people into a false sense of security. I didn’t understand how someone with only half a set of expressions could have “villain” written all over them until I met him.”
</p>
<p>
“He’s always been like that. He feigns concern and sympathy to lure in society’s outcasts. They’re easy targets,” All Might said through a mouthful of biscuit.
</p>
<p>
“Has he ever tried it on any of the One For All successors?”
</p>
<p>
“Not really, but you might have accidentally given him the incentive for it. He never had access to any of the One For All wielders while they were young.” All Might snorted, “not that it’ll make a difference with you”.
</p>
<p>
“I think he was trying to gauge me for a world view before the wardens ended it. I need more time to work out his response to the stuff on his Quirk.”
</p>
<p>
“He’s conversation starved since it’s solitary confinement. If what the people monitoring his brain activity said was true, you’re the most exciting thing to have happened to him in months. He replied after you left, said he was looking forward to it.”
</p>
<p>
“That’s pretty sad."
</p>
<p>
“It’s even sadder that we’re the only two members of the public who have had anything to do with him. Stain gets a pile of mail from his “fans”, but All For One has nothing,” All Might waved a tea spoon. “That’s what he gets.”
</p>
<p>
“Let’s get out of here and tell Detective Tsukauchi how it went.” Izuku gulped down his tea and headed for the exit, with him and All Might reaching it at roughly the same amount of time.
</p>
<p>
“At least your mum’s making katsudon for us tonight," was All Might's only optimistic comment.
</p>
<p>
Anxiety was still ebbing over Izuku after Tsukauchi had been debriefed in the car.
</p>
<p><i>“It seems we share a hobby.”</i> Haunted Izuku on the drive home. As if ripping someone’s Quirk from them and leaving them lying traumatised on the ground was just a fun pastime and not an act of grievous bodily harm.
</p>
<p>
And he’d be dealing with him again in another week.
</p>
</div></article>

1025
resources/tests/readability/archive-of-our-own/source.html Normal file
View file

File diff suppressed because one or more lines are too long

52
resources/tests/readability/ars-1/expected.html Normal file
View file

@ -0,0 +1,52 @@
<article><div id="readability-page-1">
<header><h4>
Biz &amp; IT —
</h4>
<h2 itemprop="description">
Two-year-old bug exposes thousands of servers to crippling attack.
</h2>
<section></section></header><section><div itemprop="articleBody">
<figure><img src="https://cdn.arstechnica.net/wp-content/uploads/2015/04/server-crash-640x426.jpg" alt="Just-released Minecraft exploit makes it easy to crash game servers"><figcaption></figcaption></figure><p>
A flaw in the wildly popular online game <em>Minecraft</em> makes it easy for just about anyone to crash the server hosting the game, according to a computer programmer who has released proof-of-concept code that exploits the vulnerability.
</p>
<p>
"I thought a lot before writing this post," Pakistan-based developer Ammar Askar wrote in a <a href="http://blog.ammaraskar.com/minecraft-vulnerability-advisory" target="_blank">blog post published Thursday</a>, 21 months, he said, after privately reporting the bug to <em>Minecraft</em> developer Mojang. "On the one hand I don't want to expose thousands of servers to a major vulnerability, yet on the other hand Mojang has failed to act on it."
</p>
<p>
The bug resides in the <a href="https://github.com/ammaraskar/pyCraft" target="_blank">networking internals of the <em>Minecraft</em> protocol</a>. It allows the contents of inventory slots to be exchanged, so that, among other things, items in players' hotbars are displayed automatically after logging in. <em>Minecraft</em> items can also store arbitrary metadata in a file format known as <a href="http://wiki.vg/NBT" target="_blank">Named Binary Tag (NBT)</a>, which allows complex data structures to be kept in hierarchical nests. Askar has released <a href="https://github.com/ammaraskar/pyCraft/tree/nbt_exploit" target="_blank">proof-of-concept attack code</a> he said exploits the vulnerability to crash any server hosting the game. Here's how it works.
</p>
<blockquote>
<p>
The vulnerability stems from the fact that the client is allowed to send the server information about certain slots. This, coupled with the NBT format’s nesting allows us to <em>craft</em> a packet that is incredibly complex for the server to deserialize but trivial for us to generate.
</p>
<p>
In my case, I chose to create lists within lists, down to five levels. This is a json representation of what it looks like.
</p>
<div><pre><code data-lang="javascript"><span>rekt</span><span>:</span><span>{</span><span>list</span><span>:</span><span>[</span><span>list</span><span>:</span><span>[</span><span>list</span><span>:</span><span>[</span><span>list</span><span>:</span><span>[</span><span>list</span><span>:</span><span>[</span><span>list</span><span>:</span><span>[</span><span>]</span><span>list</span><span>:</span><span>[</span><span>]</span><span>list</span><span>:</span><span>[</span><span>]</span><span>list</span><span>:</span><span>[</span><span>]</span><span>...</span><span>]</span><span>...</span><span>]</span><span>...</span><span>]</span><span>...</span><span>]</span><span>...</span><span>]</span><span>...</span><span>}</span></code></pre></div>
<p>
The root of the object, <code>rekt</code>, contains 300 lists. Each list has a list with 10 sublists, and each of those sublists has 10 of their own, up until 5 levels of recursion. That’s a total of <code>10^5 * 300 = 30,000,000</code> lists.
</p>
<p>
And this isn’t even the theoretical maximum for this attack. Just the nbt data for this payload is 26.6 megabytes. But luckily Minecraft implements a way to compress large packets, lucky us! zlib shrinks down our evil data to a mere 39 kilobytes.
</p>
<p>
Note: in previous versions of Minecraft, there was no protocol wide compression for big packets. Previously, NBT was sent compressed with gzip and prefixed with a signed short of its length, which reduced our maximum payload size to <code>2^15 - 1</code>. Now that the length is a varint capable of storing integers up to <code>2^28</code>, our potential for attack has increased significantly.
</p>
<p>
When the server will decompress our data, it’ll have 27 megs in a buffer somewhere in memory, but that isn’t the bit that’ll kill it. When it attempts to parse it into NBT, it’ll create java representations of the objects meaning suddenly, the sever is having to create several million java objects including ArrayLists. This runs the server out of memory and causes tremendous CPU load.
</p>
<p>
This vulnerability exists on almost all previous and current Minecraft versions as of 1.8.3, the packets used as attack vectors are the <a href="http://wiki.vg/Protocol#Player_Block_Placement" target="_blank">0x08: Block Placement Packet</a> and <a href="http://wiki.vg/Protocol#Creative_Inventory_Action" target="_blank">0x10: Creative Inventory Action</a>.
</p>
<p>
The fix for this vulnerability isn’t exactly that hard, the client should never really send a data structure as complex as NBT of arbitrary size and if it must, some form of recursion and size limits should be implemented.
</p>
<p>
These were the fixes that I recommended to Mojang 2 years ago.
</p>
</blockquote>
<p>
Ars is asking Mojang for comment and will update this post if company officials respond.
</p>
</div></section>
</div></article>

634
resources/tests/readability/ars-1/source.html Normal file
View file

@ -0,0 +1,634 @@
<!DOCTYPE html>
<html lang="en-us" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-us">
<head>
<title>
Just-released Minecraft exploit makes it easy to crash game servers | Ars Technica
</title>
<script type="text/javascript">
//<![CDATA[
ars = {"ASSETS":"https:\/\/cdn.arstechnica.net\/wp-content\/themes\/ars\/assets","HOME_URL":"https:\/\/arstechnica.com","LOGIN_URL":"https:\/\/arstechnica.com\/services\/login-desktop.html?v=1","CIVIS":"\/civis","THEME":"light","VIEW":"grid","MOBILE":false,"SUBSCRIBER":false,"PLUS_PLUS":false,"LOGGED":false,"USER_ID":null,"ENV":"production","AD":{"tags":["denial-of-service-attack","exploits","minecraft","vulnerabilities"],"channel":"information-technology","slug":"just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers","template_type":"article","queue":[],"server":"production"},"TOTAL":97063,"UNREAD":0,"RECENT":[1698939,1698645,1698804,1698783,1698769,1698663,1698682,1698690,1698667,1698588,1698619,1697597,1698183,1698597,1698540,1698542,1698370,1698442,1698274,1698421,1698346,1698367,1698356,1698294,1698335],"LOGINS":true,"CROSS":false,"PARSELY":"arstechnica.com","COMMENTS":false,"HOMEPAGE":false,"SITE":1,"READY":[],"SHOW_ADS":true,"IMG_PROXY":"https:\/\/cdn.arstechnica.net\/i\/","CATEGORY":"information-technology","PAGETITLE":"","ZEN_MODE":false};
//]]>
</script>
<link rel="stylesheet" type="text/css" media="all" href="https://cdn.arstechnica.net/wp-content/themes/ars/assets/css/main-130fcfcce0.css" />
<link rel="alternate" type="application/rss+xml" href="http://feeds.arstechnica.com/arstechnica/index/" />
<link rel="shortcut icon" href="https://cdn.arstechnica.net/favicon.ico" />
<link rel="icon" type="image/x-icon" href="https://cdn.arstechnica.net/favicon.ico" />
<link rel="apple-touch-icon" sizes="180x180" href="https://cdn.arstechnica.net/wp-content/themes/ars/assets/img/ars-ios-icon-d9a45f558c.png" />
<link rel="mask-icon" href="https://cdn.arstechnica.net/wp-content/themes/ars/assets/img/ars-macos-safari-8997f76b21.svg" color="#ff4e00" />
<link rel="icon" sizes="192x192" href="https://cdn.arstechnica.net/wp-content/themes/ars/assets/img/material-ars-db41652381.png" />
<meta name="application-name" content="Ars Technica" />
<meta name="msapplication-starturl" content="http://arstechnica.com/" />
<meta name="msapplication-tooltip" content="Ars Technica: Serving the technologist for 1.2 decades" />
<meta name="msapplication-task" content="name=News;action-uri=http://arstechnica.com/;icon-uri=https://cdn.arstechnica.net/favicon.ico" />
<meta name="msapplication-task" content="name=Features;action-uri=http://arstechnica.com/features/;icon-uri=https://cdn.arstechnica.net/ie-jump-menu/jump-features.ico" />
<meta name="msapplication-task" content="name=OpenForum;action-uri=http://arstechnica.com/civis/;icon-uri=https://cdn.arstechnica.net/ie-jump-menu/jump-forum.ico" />
<meta name="msapplication-task" content="name=Subscribe;action-uri=http://arstechnica.com/subscriptions/;icon-uri=https://cdn.arstechnica.net/ie-jump-menu/jump-subscribe.ico" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="advertising" content="ask" />
<meta property="fb:admins" content="592156917" />
<meta property="fb:admins" content="108943" />
<meta property="fb:pages" content="19374573752" />
<meta name="format-detection" content="telephone=no" />
<meta name="theme-color" content="#000000" />
<meta name="viewport" content="width=device-width,initial-scale=1" /><!-- cache miss 581:single/meta:5a5daf59fa5245a64fe8615caa0b1d1b -->
<meta name="parsely-page" content="{&quot;title&quot;:&quot;Just-released Minecraft exploit makes it easy to crash game servers&quot;,&quot;link&quot;:&quot;https:\/\/arstechnica.com\/information-technology\/2015\/04\/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers\/&quot;,&quot;type&quot;:&quot;post&quot;,&quot;author&quot;:&quot;Dan Goodin&quot;,&quot;post_id&quot;:648287,&quot;pub_date&quot;:&quot;2015-04-16T20:02:01Z&quot;,&quot;section&quot;:&quot;Biz &amp; IT&quot;,&quot;tags&quot;:[&quot;denial-of-service-attack&quot;,&quot;exploits&quot;,&quot;minecraft&quot;,&quot;vulnerabilities&quot;,&quot;type: report&quot;],&quot;image_url&quot;:&quot;https:\/\/cdn.arstechnica.net\/wp-content\/uploads\/2015\/04\/server-crash-150x150.jpg&quot;}" />
<meta name="parsely-metadata" content="{&quot;type&quot;:&quot;report&quot;,&quot;title&quot;:&quot;Just-released Minecraft exploit makes it easy to crash game servers&quot;,&quot;post_id&quot;:648287,&quot;lower_deck&quot;:&quot;Two-year-old bug exposes thousands of servers to crippling attack.&quot;,&quot;image_url&quot;:&quot;https:\/\/cdn.arstechnica.net\/wp-content\/uploads\/2015\/04\/server-crash-150x150.jpg&quot;,&quot;listing_image_url&quot;:&quot;https:\/\/cdn.arstechnica.net\/wp-content\/uploads\/2015\/04\/server-crash-300x150.jpg&quot;}" />
<link rel="canonical" href="https://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/" />
<link rel="amphtml" href="https://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/?amp=1" />
<link rel="shorturl" href="https://arstechnica.com/?p=648287" />
<meta name="description" content="Two-year-old bug exposes thousands of servers to crippling attack." />
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:url" content="https://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/" />
<meta name="twitter:title" content="Just-released Minecraft exploit makes it easy to crash game servers" />
<meta name="twitter:description" content="Two-year-old bug exposes thousands of servers to crippling attack." />
<meta name="twitter:site" content="@arstechnica" />
<meta name="twitter:domain" content="arstechnica.com" />
<meta property="og:site_name" content="Ars Technica" />
<meta name="twitter:image:src" content="https://cdn.arstechnica.net/wp-content/uploads/2015/04/server-crash-640x215.jpg" />
<meta name="twitter:image:width" content="640" />
<meta name="twitter:image:height" content="215" />
<meta name="twitter:creator" content="@dangoodin001" />
<meta property="og:url" content="https://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/" />
<meta property="og:title" content="Just-released Minecraft exploit makes it easy to crash game servers" />
<meta property="og:image" content="https://cdn.arstechnica.net/wp-content/uploads/2015/04/server-crash-640x215.jpg" />
<meta property="og:description" content="Two-year-old bug exposes thousands of servers to crippling attack." />
<meta property="og:type" content="article" /><!-- cache hit 581:single/header:5a5daf59fa5245a64fe8615caa0b1d1b -->
<!-- Google Tag Manager DataLayer -->
<script>
<![CDATA[
window.dataLayer = window.dataLayer || [];
window.dataLayer.push({"event":"data-layer-loaded","user":{"ars_userId":undefined,"amg_userId":undefined,"uID":undefined,"sID":undefined,"loginStatus":false,"subscriberStatus":"none","infinityId":undefined,"registrationSource":undefined,"mdw_cnd_id":undefined,"monthlyVisits":undefined,"accessPaywall":undefined,"view":"grid","theme":"light","show_comments":false},"content":{"pageTemplate":"single","pageType":"article|report","contentCategory":"information-technology","section":"information technology","subsection":undefined,"contributor":"Dan Goodin","contentID":648287,"contentLength":835,"display":"Just-released Minecraft exploit makes it easy to crash game servers","contentSource":"web","pageAssets":undefined,"uniqueContentCount":undefined,"monthlyContentCount":undefined,"publishDate":"2015-04-16T20:02:01+00:00","modifiedDate":"2015-04-16T20:11:02+00:00","keywords":"denial of service attack|exploits|minecraft|vulnerabilities","dataSource":undefined},"marketing":{"campaignName":undefined,"circCampaignId":undefined,"internalCampaignId":undefined,"brand":"Ars Technica","certified_mrc_data":undefined,"condeNastId":undefined},"page":{"pID":undefined,"syndicatorUrl":undefined,"pageURL":"https:\/\/arstechnica.com\/?p=648287","canonical":"https:\/\/arstechnica.com\/information-technology\/2015\/04\/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers\/","canonicalPathName":"\/information-technology\/2015\/04\/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers\/"},"search":{"facets":undefined,"searchTerms":undefined},"site":{"appVersion":"1.0.0"}});
]]>
</script><!-- End Google Tag Manager DataLayer -->
<!-- Google Tag Manager -->
<script>
<![CDATA[
(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-NLXNPCQ');
]]>
</script><!-- End Google Tag Manager -->
<!-- OneTrust Cookies Consent Notice start -->
<script src="https://cdn.cookielaw.org/scripttemplates/otSDKStub.js" type="text/javascript" charset="UTF-8" data-domain-script="b10882a1-8446-4e7d-bfb2-ce2c770ad910"></script>
<script type="text/javascript">
//<![CDATA[
function OptanonWrapper(){};
//]]>
</script>
<script src="https://cdn.cookielaw.org/opt-out/otCCPAiab.js" type="text/javascript" charset="UTF-8" ccpa-opt-out-ids="C0002,C0003,C0004,C0005" ccpa-opt-out-geo="ca" ccpa-opt-out-lspa="true"></script><!-- OneTrust Cookies Consent Notice end -->
<script src="https://www.googletagservices.com/tag/js/gpt.js" id="gpt-script" async="async"></script>
<script>
<![CDATA[
window.googletag=window.googletag||{};window.googletag.cmd=window.googletag.cmd||[];window.cns=window.cns||{};window.cns.queue=[];window.cns.async=function(s,c){cns.queue.push({service:s,callback:c})};window.sparrowQueue=window.sparrowQueue||[];
]]>
</script>
<link rel="dns-prefetch" href="//aax.amazon-adsystem.com" />
<link rel="preconnect" href="//aax.amazon-adsystem.com" crossorigin="" />
<link rel="preconnect" href="https://mb.moatads.com" crossorigin="" />
<script src="https://c.amazon-adsystem.com/aax2/apstag.js" async="async"></script>
<script src="https://cdn.arstechnica.net/cns/prebid.min.js?v=1597375105"></script>
<script src="https://js-sec.indexww.com/ht/p/183973-93942139695505.js" async="async"></script>
<script src="https://z.moatads.com/condenastprebidheader987326845656/moatheader.js" async="async"></script>
<script>
<![CDATA[
window.cns.pageContext = {"contentType":"article","templateType":"article","channel":"information-technology","subChannel":"","slug":"just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers","server":"production","keywords":{"tags":["denial-of-service-attack","exploits","minecraft","vulnerabilities"],"cm":[],"platform":["wordpress"],"copilotid":""}};
]]>
</script>
<script src="https://cdn.arstechnica.net/cns/ars-technica.min.js?v=1597375105"></script>
<script type="text/javascript" src="https://cdn.arstechnica.net/wp-content/themes/ars/assets/js/ars-32ecec341f.ads.us.js"></script>
</head>
<body class="post-template-default single single-post postid-648287 single-format-standard grid-view light blog-us">
<!-- Google Tag Manager (noscript) -->
<noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-NLXNPCQ" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript> <!-- End Google Tag Manager (noscript) -->
<aside class="ad ad_crown" aria-label="Top of page advertisement"></aside>
<div class="site-wrapper">
<a class="screen-reader-text skip-link" href="#main" aria-label="Skip to main content">Skip to main content</a>
<header class="site-header">
<div class="header-left">
<a href="https://arstechnica.com" id="header-logo" title="Ars Technica Homepage"></a>
</div>
<div class="header-right">
<nav id="header-nav-primary">
<ul>
<li>
<a class="nav-link section-information-technology active" href="/information-technology/">Biz &amp; IT</a>
</li>
<li>
<a class="nav-link section-gadgets" href="/gadgets/">Tech</a>
</li>
<li>
<a class="nav-link section-science" href="/science/">Science</a>
</li>
<li>
<a class="nav-link section-tech-policy" href="/tech-policy/">Policy</a>
</li>
<li>
<a class="nav-link section-cars" href="/cars/">Cars</a>
</li>
<li>
<a class="nav-link section-gaming" href="/gaming/">Gaming &amp; Culture</a>
</li>
<li>
<a class="nav-link store" href="/store/">Store</a>
</li>
<li>
<a class="nav-link forums" href="/civis/">Forums</a>
</li>
</ul>
</nav><a href="/store/product/subscriptions/" class="header-highlight-link">Subscribe</a>
<div class="dropdown" id="header-search">
<a href="/search/" class="dropdown-toggle search-toggle" aria-label="Search" aria-expanded="false"></a>
<div class="dropdown-content">
<form action="/search/" method="get" id="search_form" name="search_form">
<input type="hidden" name="ie" value="UTF-8" /> <input type="text" name="q" id="hdr_search_input" value="" aria-label="Search..." placeholder="Search..." />
</form><a class="nav-search-close">Close</a>
</div>
</div>
<div class="dropdown dropdown-mega" id="header-burger">
<a href="#site-menu" class="dropdown-toggle" aria-label="Menu" aria-expanded="false"></a>
<div id="site-menu" class="dropdown-content">
<section class="burger-navigate">
<h3>
Navigate
</h3>
<ul>
<li>
<a class="nav-link store" href="/store/">Store</a>
</li>
<li>
<a class="nav-link subscribe" href="/store/product/subscriptions/">Subscribe</a>
</li>
<li>
<a class="nav-link videos" href="http://video.arstechnica.com/">Videos</a>
</li>
<li>
<a class="nav-link section-features" href="/features/">Features</a>
</li>
<li>
<a class="nav-link section-reviews" href="/reviews/">Reviews</a>
</li>
</ul>
<ul>
<li>
<a class="nav-link page-rss-feeds" href="/rss-feeds/">RSS Feeds</a>
</li>
<li>
<a class="nav-link mobile" href="/?view=mobile">Mobile Site</a>
</li>
</ul>
<ul>
<li>
<a class="nav-link page-about-us" href="/about-us/">About Ars</a>
</li>
<li>
<a class="nav-link page-staff-directory" href="/staff-directory/">Staff Directory</a>
</li>
<li>
<a class="nav-link page-contact-us" href="/contact-us/">Contact Us</a>
</li>
</ul>
<ul>
<li>
<a class="nav-link page-advertise-with-us" href="/advertise-with-us/">Advertise with Ars</a>
</li>
<li>
<a class="nav-link page-reprints" href="/reprints/">Reprints</a>
</li>
</ul>
</section>
<section class="burger-filter">
<h3>
Filter by topic
</h3>
<ul id="burger-nav-primary">
<li>
<a class="nav-link section-information-technology active" href="/information-technology/">Biz &amp; IT</a>
</li>
<li>
<a class="nav-link section-gadgets" href="/gadgets/">Tech</a>
</li>
<li>
<a class="nav-link section-science" href="/science/">Science</a>
</li>
<li>
<a class="nav-link section-tech-policy" href="/tech-policy/">Policy</a>
</li>
<li>
<a class="nav-link section-cars" href="/cars/">Cars</a>
</li>
<li>
<a class="nav-link section-gaming" href="/gaming/">Gaming &amp; Culture</a>
</li>
<li>
<a class="nav-link store" href="/store/">Store</a>
</li>
<li>
<a class="nav-link forums" href="/civis/">Forums</a>
</li>
</ul>
</section>
<section class="burger-settings">
<h3>
Settings
</h3>
<div>
<div class="burger-layout">
<p>
Front page layout
</p>
<div class="burger-layout-grid">
<a rel="nofollow" href="http://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/?view=grid" class=""><br />
Grid
<div class="faux-radio active"></div></a>
</div>
<div class="burger-layout-list">
<a rel="nofollow" href="http://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/?view=archive" class=""><br />
List
<div class="faux-radio"></div></a>
</div>
</div>
<div class="burger-theme">
<p>
Site theme
</p>
<div class="burger-theme-light">
<a rel="nofollow" href="http://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/?theme=light" class=""><span><span>Black on white</span></span>
<div class="faux-radio active"></div></a>
</div>
<div class="burger-theme-dark">
<a rel="nofollow" href="http://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/?theme=dark" class=""><span><span>White on black</span></span>
<div class="faux-radio"></div></a>
</div>
</div>
</div>
</section>
</div>
</div>
<div class="dropdown dropdown-mega" id="header-account">
<a href="https://arstechnica.com/civis/ucp.php?mode=login&amp;return_to=%2Finformation-technology%2F2015%2F04%2Fjust-released-minecraft-exploit-makes-it-easy-to-crash-game-servers%2F" class="dropdown-toggle" aria-expanded="false">Sign in</a>
<div class="dropdown-content">
<section class="profile-activity">
<h3>
Comment activity
</h3>
<p>
Sign up or login to join the discussions!
</p>
</section>
<section class="profile-settings">
<form id="login-form" action="https://arstechnica.com/civis/ucp.php?mode=login" method="post" name="login-form">
<input type="text" name="username" id="username" placeholder="Username or Email" aria-label="Username or Email" /> <input type="password" name="password" id="password" placeholder="Password" aria-label="Password" /> <input type="submit" value="Submit" class="button button-orange button-wide" name="login" /> <label id="remember-label"><input type="checkbox" name="autologin" id="autologin" /> Stay logged in</label> <span>|</span> <a href="/civis/ucp.php?mode=sendpassword">Having trouble?</a> <input type="hidden" name="redirect" value="./ucp.php?mode=login&amp;autoredirect=1&amp;return_to=%2Finformation-technology%2F2015%2F04%2Fjust-released-minecraft-exploit-makes-it-easy-to-crash-game-servers%2F" /> <input type="hidden" name="return_to" value="/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/" />
</form>
<div class="register-account">
<span>Sign up to comment and more</span> <a href="https://arstechnica.com/civis/ucp.php?mode=register" class="signup-btn button button-wide">Sign up</a>
</div>
</section>
</div>
</div>
</div>
</header>
<main id="main" class="content-wrapper">
<script type="text/javascript">
//<![CDATA[
ars.ARTICLE = {"url":"https:\/\/arstechnica.com\/information-technology\/2015\/04\/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers\/","short_url":"https:\/\/arstechnica.com\/?p=648287","title":"Just-released Minecraft exploit makes it easy to crash game servers","author":329388,"id":648287,"topic":1280621,"pages":1,"current_page":1,"superscroll":false,"promoted":[],"single_page":false,"comments":75,"fullwidth":false,"slug":"just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers","arsStaff":{"104481":{"name":"Aaron Zimmerman","title":"Copyeditor","staff":true},"1002":{"name":"Aurich Lawson","title":"Creative Director","staff":true},"509873":{"name":"Beth Mole","title":"Health Reporter","staff":true},"453791":{"name":"Cathleen O'Grady","title":"Contributing science reporter","staff":true},"102179":{"name":"Chris Lee","title":"Associate writer","staff":true},"821742":{"name":"Corey Gaskin","title":"Senior Commerce Writer","staff":true},"329388":{"name":"Dan Goodin","title":"Security Editor","staff":true},"254631":{"name":"Diana Gitig","title":"Associate Writer","staff":false},"25862":{"name":"Eric Bangeman","title":"Managing Editor","staff":true},"512413":{"name":"Eric Berger","title":"Senior Space Editor","staff":true},"46707":{"name":"Iljitsch van Beijnum","title":"Associate Writer","staff":false},"316010":{"name":"Jason Marlin","title":"Technical Director","staff":true},"746799":{"name":"Jennifer Ouellette","title":"Senior Writer","staff":true},"15365":{"name":"Jeremy Reimer","title":"Senior Niche Technology Historian","staff":false},"4086":{"name":"Jim Salter","title":"Technology Reporter","staff":true},"52979":{"name":"John Timmer","title":"Senior Science Editor","staff":true},"312082":{"name":"Jon Brodkin","title":"Senior IT Reporter","staff":true},"14317":{"name":"Jonathan M. Gitlin","title":"Automotive Editor","staff":true},"786739":{"name":"Kate Cox","title":"Tech Policy Reporter","staff":true},"998":{"name":"Ken Fisher","title":"Editor in Chief","staff":true},"440179":{"name":"Kerry Staurseth","title":"Associate Copyeditor","staff":true},"328283":{"name":"Kyle Orland","title":"Senior Gaming Editor","staff":true},"10243":{"name":"Lee Hutchinson","title":"Senior Technology Editor","staff":true},"173191":{"name":"Matthew Lasar","title":"Associate writer","staff":true},"182268":{"name":"Nate Anderson","title":"Deputy Editor","staff":true},"330533":{"name":"Nathan Mattise","title":"Features Editor","staff":true},"1991":{"name":"Ohrmazd","title":"","staff":false},"391727":{"name":"Ron Amadeo","title":"Reviews Editor","staff":true},"348927":{"name":"Sam Machkovech","title":"Tech Culture Editor","staff":true},"588289":{"name":"Samuel Axon","title":"Senior Reviews Editor","staff":true},"294205":{"name":"Scott K. Johnson","title":"Associate Writer","staff":true},"671621":{"name":"Steven Klein","title":"Developer","staff":false},"173910":{"name":"Timothy B. Lee","title":"Senior tech policy reporter","staff":true}},"tags":["denial-of-service-attack","exploits","minecraft","vulnerabilities"],"zen_mode":false,"vote_sentiments":[{"sentiment_id":"1","sentiment":"agree","direction":"positive","icon":null,"label":"Agree"},{"sentiment_id":"3","sentiment":"interesting","direction":"positive","icon":null,"label":"Interesting"},{"sentiment_id":"5","sentiment":"funny","direction":"positive","icon":null,"label":"Funny"},{"sentiment_id":"6","sentiment":"addsto","direction":"positive","icon":null,"label":"Adds to Story"},{"sentiment_id":"7","sentiment":"disagree","direction":"negative","icon":null,"label":"Disagree"},{"sentiment_id":"8","sentiment":"inaccurate","direction":"negative","icon":null,"label":"Inaccurate"},{"sentiment_id":"11","sentiment":"pointless","direction":"negative","icon":null,"label":"Doesn't Contribute"},{"sentiment_id":"12","sentiment":"abusive","direction":"negative","icon":null,"label":"Abusive"}]};
//]]>
</script>
<article itemscope="itemscope" itemtype="http://schema.org/NewsArticle" class="article-single standalone intro-default" id="">
<div class="column-wrapper">
<div class="left-column">
<header class="article-header">
<h4 class="post-upperdek">
Biz &amp; IT —
</h4>
<h1 itemprop="headline">
Just-released <i>Minecraft</i> exploit makes it easy to crash game servers
</h1>
<h2 itemprop="description">
Two-year-old bug exposes thousands of servers to crippling attack.
</h2>
<section class="post-meta">
<p class="byline" itemprop="author creator" itemscope="itemscope" itemtype="http://schema.org/Person">
<a itemprop="url" href="https://arstechnica.com/author/dan-goodin/" rel="author"><span itemprop="name">Dan Goodin</span></a> - <time class="date" data-time="1429214521" datetime="2015-04-16T20:02:01+00:00">Apr 16, 2015 8:02 pm UTC</time>
</p>
</section>
</header>
<section class="article-guts">
<div itemprop="articleBody" class="article-content post-page">
<figure class="intro-image intro-left">
<img src="https://cdn.arstechnica.net/wp-content/uploads/2015/04/server-crash-640x426.jpg" alt="Just-released Minecraft exploit makes it easy to crash game servers" />
<figcaption class="caption">
<div class="caption-credit">
<a rel="nofollow" class="caption-link" href="https://en.wikipedia.org/wiki/Kernel_panic#/media/File:Kernel-panic.jpg">Kevin</a>
</div>
</figcaption>
</figure>
<aside id="social-left" class="social-left" aria-label="Read the comments or share this article">
<a title="51 posters participating" class="comment-count icon-comment-bubble-down" href="https://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/?comments=1">
<h4 class="comment-count-before">
reader comments
</h4><span class="comment-count-number">75</span> <span class="visually-hidden">with 51 posters participating</span></a>
<div class="share-links">
<h4>
Share this story
</h4>
<ul>
<li>
<a href="https://www.facebook.com/sharer.php?u=https%3A%2F%2Farstechnica.com%2F%3Fpost_type%3Dpost%26p%3D648287" target="_blank" class="social-icon share-facebook" title="Share on Facebook"><span class="visually-hidden">Share on Facebook</span></a>
</li>
<li>
<a href="https://twitter.com/share?text=Just-released+%3Ci%3EMinecraft%3C%2Fi%3E+exploit+makes+it+easy+to+crash+game+servers&amp;url=https%3A%2F%2Farstechnica.com%2F%3Fpost_type%3Dpost%26p%3D648287" target="_blank" class="social-icon share-twitter" title="Share on Twitter"><span class="visually-hidden">Share on Twitter</span></a>
</li>
<li>
<a href="https://www.reddit.com/submit?url=https%3A%2F%2Farstechnica.com%2F%3Fpost_type%3Dpost%26p%3D648287&amp;title=Just-released+%3Ci%3EMinecraft%3C%2Fi%3E+exploit+makes+it+easy+to+crash+game+servers" target="_blank" class="social-icon share-reddit" title="Share on Reddit"><span class="visually-hidden">Share on Reddit</span></a>
</li>
</ul>
</div>
</aside><!-- cache miss 581:single/related:5a5daf59fa5245a64fe8615caa0b1d1b --><!-- empty -->
<p>
A flaw in the wildly popular online game <em>Minecraft</em> makes it easy for just about anyone to crash the server hosting the game, according to a computer programmer who has released proof-of-concept code that exploits the vulnerability.
</p>
<p>
"I thought a lot before writing this post," Pakistan-based developer Ammar Askar wrote in a <a href="http://blog.ammaraskar.com/minecraft-vulnerability-advisory">blog post published Thursday</a>, 21 months, he said, after privately reporting the bug to <em>Minecraft</em> developer Mojang. "On the one hand I don't want to expose thousands of servers to a major vulnerability, yet on the other hand Mojang has failed to act on it."
</p>
<p>
The bug resides in the <a href="https://github.com/ammaraskar/pyCraft">networking internals of the <em>Minecraft</em> protocol</a>. It allows the contents of inventory slots to be exchanged, so that, among other things, items in players' hotbars are displayed automatically after logging in. <em>Minecraft</em> items can also store arbitrary metadata in a file format known as <a href="http://wiki.vg/NBT">Named Binary Tag (NBT)</a>, which allows complex data structures to be kept in hierarchical nests. Askar has released <a href="https://github.com/ammaraskar/pyCraft/tree/nbt_exploit">proof-of-concept attack code</a> he said exploits the vulnerability to crash any server hosting the game. Here's how it works.
</p>
<blockquote>
<p>
The vulnerability stems from the fact that the client is allowed to send the server information about certain slots. This, coupled with the NBT format’s nesting allows us to <em>craft</em> a packet that is incredibly complex for the server to deserialize but trivial for us to generate.
</p>
<p>
In my case, I chose to create lists within lists, down to five levels. This is a json representation of what it looks like.
</p>
<div class="highlight">
<pre><code class="language-javascript" data-lang="javascript"><span class="nx">rekt</span><span class="o">:</span> <span class="p">{</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="p">]</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="p">]</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="p">]</span>
<span class="nx">list</span><span class="o">:</span> <span class="p">[</span>
<span class="p">]</span>
<span class="p">...</span>
<span class="p">]</span>
<span class="p">...</span>
<span class="p">]</span>
<span class="p">...</span>
<span class="p">]</span>
<span class="p">...</span>
<span class="p">]</span>
<span class="p">...</span>
<span class="p">]</span>
<span class="p">...</span>
<span class="p">}</span></code></pre>
</div>
<p>
The root of the object, <code>rekt</code>, contains 300 lists. Each list has a list with 10 sublists, and each of those sublists has 10 of their own, up until 5 levels of recursion. That’s a total of <code>10^5 * 300 = 30,000,000</code> lists.
</p>
<p>
And this isn’t even the theoretical maximum for this attack. Just the nbt data for this payload is 26.6 megabytes. But luckily Minecraft implements a way to compress large packets, lucky us! zlib shrinks down our evil data to a mere 39 kilobytes.
</p>
<p>
Note: in previous versions of Minecraft, there was no protocol wide compression for big packets. Previously, NBT was sent compressed with gzip and prefixed with a signed short of its length, which reduced our maximum payload size to <code>2^15 - 1</code>. Now that the length is a varint capable of storing integers up to <code>2^28</code>, our potential for attack has increased significantly.
</p>
<p>
When the server will decompress our data, it’ll have 27 megs in a buffer somewhere in memory, but that isn’t the bit that’ll kill it. When it attempts to parse it into NBT, it’ll create java representations of the objects meaning suddenly, the sever is having to create several million java objects including ArrayLists. This runs the server out of memory and causes tremendous CPU load.
</p>
<p>
This vulnerability exists on almost all previous and current Minecraft versions as of 1.8.3, the packets used as attack vectors are the <a href="http://wiki.vg/Protocol#Player_Block_Placement">0x08: Block Placement Packet</a> and <a href="http://wiki.vg/Protocol#Creative_Inventory_Action">0x10: Creative Inventory Action</a>.
</p>
<p>
The fix for this vulnerability isn’t exactly that hard, the client should never really send a data structure as complex as NBT of arbitrary size and if it must, some form of recursion and size limits should be implemented.
</p>
<p>
These were the fixes that I recommended to Mojang 2 years ago.
</p>
</blockquote>
<p>
Ars is asking Mojang for comment and will update this post if company officials respond.
</p>
<div id="action_button_container"></div>
</div>
</section>
</div>
<div class="xrail">
<div class="xrail-content">
<aside class="ad ad_xrail ad_xrail_top" aria-label="Top sidebar advertisement"></aside>
<aside class="ad_native ad_native_xrail" aria-label="Sidebar native advertisement"></aside>
</div>
</div>
</div>
<div class="column-wrapper">
<div class="left-column">
<div id="social-footer">
<a title="51 posters participating" class="comment-count icon-comment-bubble-down" href="https://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/?comments=1">
<h4 class="comment-count-before">
reader comments
</h4><span class="comment-count-number">75</span> <span class="visually-hidden">with 51 posters participating</span></a>
<div class="share-links">
<h4>
Share this story
</h4>
<ul>
<li>
<a href="https://www.facebook.com/sharer.php?u=https%3A%2F%2Farstechnica.com%2F%3Fpost_type%3Dpost%26p%3D648287" target="_blank" class="social-icon share-facebook" title="Share on Facebook"><span class="visually-hidden">Share on Facebook</span></a>
</li>
<li>
<a href="https://twitter.com/share?text=Just-released+%3Ci%3EMinecraft%3C%2Fi%3E+exploit+makes+it+easy+to+crash+game+servers&amp;url=https%3A%2F%2Farstechnica.com%2F%3Fpost_type%3Dpost%26p%3D648287" target="_blank" class="social-icon share-twitter" title="Share on Twitter"><span class="visually-hidden">Share on Twitter</span></a>
</li>
<li>
<a href="https://www.reddit.com/submit?url=https%3A%2F%2Farstechnica.com%2F%3Fpost_type%3Dpost%26p%3D648287&amp;title=Just-released+%3Ci%3EMinecraft%3C%2Fi%3E+exploit+makes+it+easy+to+crash+game+servers" target="_blank" class="social-icon share-reddit" title="Share on Reddit"><span class="visually-hidden">Share on Reddit</span></a>
</li>
</ul>
</div>
</div><!-- cache hit 581:single/author:2814756d09510ff24ad530ca37a5a9a9 -->
<section class="article-author">
<a style="background-image:url('https://cdn.arstechnica.net/wp-content/uploads/2018/10/Dang.jpg');" class="author-photo" href="/author/dan-goodin" tabindex="-1" role="presentation" aria-hidden="true"></a>
<div class="author-bio">
<section class="author-bio-top">
<a href="/author/dan-goodin" class="author-name">Dan Goodin</a> Dan is the Security Editor at Ars Technica, which he joined in 2012 after working for The Register, the Associated Press, Bloomberg News, and other publications.
</section>
<section class="author-social">
<strong>Email</strong> <a href="mailto:dan.goodin@arstechnica.com">dan.goodin@arstechnica.com</a> <span class="slashes">//</span> <strong>Twitter</strong> <a href="https://www.twitter.com/dangoodin001" target="_blank">@dangoodin001</a>
</section>
</div>
</section>
</div>
<div class="xrail"></div>
</div>
<div id="article-footer-wrap">
<aside class="ad ad_fullwidth fullwidth" aria-label="Full width advertisement"></aside>
<section id="comments-area" class="comments-area column-wrapper">
<div class="row comments-row left-column">
<a name="comments-bar" id="comments-bar"></a>
<div id="comments-container"></div>
<div id="comments-posting-container" class="thick-divide-bottom">
<p id="reply">
You must <a href="https://arstechnica.com/civis/ucp.php?mode=login&amp;return_to/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/" class="vote_login">login or create an account</a> to comment.
</p>
</div>
</div>
<div class="xrail xrail-comments">
<div class="xrail-content xrail-content-comments">
<aside class="ad ad_xrail ad_xrail_comments" aria-label="Comments sidebar advertisement"></aside>
</div>
</div>
</section>
<section class="inline-playlist">
<div class="ars-video-playlist">
<h3 class="ars-video-playlist-module-header">
Channel <span>Ars Technica</span>
</h3>
<div class="ars-video-playlist-module" data-playlist-id="arstechnica-channel-ars-information-technology" data-video-options="[]"></div>
</div>
</section>
<div class="prev-next-links">
<a href="https://arstechnica.com/tech-policy/2015/04/dozens-of-us-government-online-whistleblower-sites-not-secured-by-https/" rel="prev"><span class="arrow">←</span> Previous story</a> <a href="https://arstechnica.com/gaming/2015/04/hidden-files-suggest-street-fighters-ryu-may-come-to-smash-bros/" rel="next">Next story <span class="arrow">→</span></a>
</div>
<footer id="article-footer">
<div id="recommendations-footer">
<div id="story-recommendations">
<div class="heading-column">
<h3>
Related Stories
</h3>
</div>
<ul id="story-recs" class="rec-wrap"></ul>
</div>
<div id="sponsored-recommendations">
<div class="heading-column">
<h3>
Sponsored Stories
</h3><a href="http://www.outbrain.com/what-is/default/en" target="_blank">Powered by </a>
</div>
<ul id="outbrain-recs"></ul>
</div>
<div id="latest-stories">
<div class="heading-column">
<h3>
Today on Ars
</h3>
</div>
<ul id="latest-recs" class="rec-wrap"></ul>
</div>
</div>
</footer>
</div>
</article>
</main>
<footer class="site-footer">
<nav class="nav-footer">
<section>
<ul>
<li>
<a href="/store/">Store</a>
</li>
<li>
<a href="/store/product/subscriptions/">Subscribe</a>
</li>
<li>
<a href="/about-us/">About Us</a>
</li>
<li>
<a href="/rss-feeds/">RSS Feeds</a>
</li>
<li>
<a rel="nofollow" href="http://arstechnica.com/information-technology/2015/04/just-released-minecraft-exploit-makes-it-easy-to-crash-game-servers/?view=mobile">View Mobile Site</a>
</li>
</ul>
</section>
<section>
<ul>
<li>
<a href="/contact-us/">Contact Us</a>
</li>
<li>
<a href="/staff-directory/">Staff</a>
</li>
<li>
<a href="/advertise-with-us/">Advertise with us</a>
</li>
<li>
<a href="/reprints/">Reprints</a>
</li>
</ul>
</section>
<section class="footer-newsletter">
<div class="newsletter-wrapper">
<h3>
<a href="/newsletters/">Newsletter Signup</a>
</h3>
<p>
Join the Ars Orbital Transmission mailing list to get weekly updates delivered to your inbox.
</p><a class="button" href="/newsletters/">Sign me up →</a>
</div>
</section>
</nav>
<section class="footer-terms-logo">
<div class="cn-logo">
<a href="http://condenast.com/" class="icon icon-logo-cn-us" title="Visit Condé Nast"></a>
</div>
<p id="copyright-terms">
CNMN Collection<br />
WIRED Media Group<br />
© 2020 Condé Nast. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our <a href="https://www.condenast.com/user-agreement/">User Agreement</a> (updated 1/1/20) and <a href="https://www.condenast.com/privacy-policy/">Privacy Policy and Cookie Statement</a> (updated 1/1/20) and <a href="/amendment-to-conde-nast-user-agreement-privacy-policy/">Ars Technica Addendum</a> (effective 8/21/2018). Ars may earn compensation on sales from links on this site. <a href="/affiliate-link-policy/">Read our affiliate link policy</a>.<br />
<a href="https://www.condenast.com/privacy-policy/#california">Your California Privacy Rights</a> | <a id="ot-sdk-btn" class="ot-sdk-show-settings">Do Not Sell My Personal Information</a><br />
The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Condé Nast.<br />
<a href="https://www.condenast.com/online-behavioral-advertising-oba-and-how-to-opt-out-of-oba/#clickheretoreadmoreaboutonlinebehavioraladvertising(oba)">Ad Choices</a>
</p>
</section>
</footer>
</div>
<script type="text/javascript" src="https://cdn.arstechnica.net/wp-content/themes/ars/assets/js/main-fafdd8b108.js"></script> <!-- cache hit 581:single/javascript-footer:5a5daf59fa5245a64fe8615caa0b1d1b -->
<script async="async" type="application/javascript" src="https://embed.actionbutton.co/widget/widget.min.js"></script> <!-- Parse.ly start -->
<script id="parsely-cfg" src="//fpa-cdn.arstechnica.com/keys/arstechnica.com/p.js"></script> <!-- Parse.ly end -->
<script src="https://player.cnevids.com/interlude/arstechnica.js" async="async"></script>
<script id="conde-polar" src="https://cdn.mediavoice.com/nativeads/script/condenastcorporate/conde-asa-polar-master.js" async="async"></script>
<script>
<![CDATA[
(function () {
function DQ() {
var queue = window.sparrowQueue;
this.push = fn => fn();
window.sparrowQueue = this;
while (queue.length) {
queue.shift()();
}
}
function e(t, e) {
var n, a, o;
a = !1, n = document.createElement("script"), n.type = "text/javascript", n.src = t, n.onload = n.onreadystatechange = function () {
a || this.readyState && "complete" != this.readyState || (a = !0, e ? e() : !0)
}, o = document.getElementsByTagName("script")[0], o.parentNode.insertBefore(n, o)
}
if (location.search.indexOf('no_sparrow') < 0) {
e("https://pixel.condenastdigital.com/config/v2/production/ars-technica.config.js", function () {
e("https://pixel.condenastdigital.com/sparrow.min.js", function () {
if (window.SparrowConfigV2) {
window.sparrow = new window.Sparrow(window.SparrowConfigV2);
new DQ();
}
})
})
}
})();
]]>
</script>
<script type="text/javascript" src="//s.skimresources.com/js/100098X1555750.skimlinks.js"></script>
</body>
</html>

11
src/full_text_parser/readability/tests.rs
View file

@ -75,6 +75,17 @@ async fn aktualne() {
run_test("aktualne").await run_test("aktualne").await
} }
#[tokio::test]
async fn archive_of_our_own() {
run_test("archive-of-our-own").await
}
#[tokio::test]
async fn ars_1() {
run_test("ars-1").await
}
#[tokio::test] #[tokio::test]
async fn webmd_1() { async fn webmd_1() {
run_test("webmd-1").await run_test("webmd-1").await