misc-mirrors/article_scraper
1
0
Fork 0
mirror of https://gitlab.com/news-flash/article_scraper.git synced 2025-07-07 16:15:32 +02:00
Code Releases Activity

4 more test & remove share elements

Browse source
This commit is contained in:
Jan Lukas Gernert 2023-04-01 17:19:37 +02:00
parent be6e08bd6d
commit 0d6db710e8
13 changed files with 5151 additions and 1110 deletions
Download patch file Download diff file Expand all files Collapse all files

1108
expected.html
View file

File diff suppressed because it is too large Load diff

216
resources/tests/readability/salon-1/expected.html Normal file
View file

@ -0,0 +1,216 @@
<article><DIV id="readability-page-1">
<p>Horror stories about the increasingly unpopular taxi service Uber have
been commonplace in recent months, but there is still much to be learned
from its handling of the recent hostage drama in downtown Sydney, Australia.
We’re told that we reveal our true character in moments of crisis, and
apparently that’s as true for companies as it is for individuals.</p>
<p>A number of experts have challenged the idea that the horrific explosion
of violence in a Sydney café was “terrorism,” since the attacker was mentally
unbalanced and acted alone. But, terror or not, the ordeal was certainly
terrifying. Amid the chaos and uncertainty, the city believed itself to
be under a coordinated and deadly attack.</p>
<p>Uber had an interesting, if predictable, response to the panic and mayhem:
It raised prices. A lot.</p>
<p>In case you missed the story, the facts are these: Someone named Man Haron
Monis, who was considered mentally unstable and had been investigated for
murdering his ex-wife, seized hostages in a café that was located in Sydney’s
Central Business District or “CBD.” In the process he put up an Islamic
flag – “igniting,” as <a href="http://www.reuters.com/article/2014/12/15/us-australia-security-idUSKBN0JS0WX20141215" target="_blank">Reuters</a> reported,
“fears of a jihadist attack in the heart of the country’s biggest city.”</p>
<p>In the midst of the fear, Uber stepped in and tweeted this announcement: 
<span>“We are all concerned with events in CBD. Fares have increased to encourage
more drivers to come online &amp; pick up passengers in the area.”</span>
</p>
<p>As <a href="http://mashable.com/2014/12/14/uber-sydney-surge-pricing/" target="_blank">Mashable </a>reports,
the company announced that it would charge a minimum of $100 Australian
to take passengers from the area immediately surrounding the ongoing crisis,
and prices increased by as much as four times the standard amount. A firestorm
of criticism quickly erupted – “<a href="https://twitter.com/Uber_Sydney" target="_blank">@Uber_Sydney</a> stop
being assholes,” one Twitter response began – and Uber soon found itself
offering free rides out of the troubled area instead.</p>
<p>That opener suggests that Uber, as part of a community under siege, is
preparing to respond in a civic manner.<em></em>
</p>
<p><em>“… Fares have increased to encourage more drivers to come online &amp; pick up passengers in the area.”</em>
</p>
<div data-toggle-group="story-13850779">
<p>But, despite the expression of shared concern, there is no sense of <em>civitas</em> to
be found in the statement that follows. There is only a transaction, executed
at what the corporation believes to be market value. Lesson #1 about Uber
is, therefore, that in its view there is no heroism, only self-interest.
This is Ayn Rand’s brutal, irrational and primitive philosophy in its purest
form: altruism is evil, and self-interest is the only true heroism.<em></em>
</p>
<p>There was once a time when we might have read of “hero cabdrivers” or
“hero bus drivers” placing themselves in harm’s way to rescue their fellow
citizens. For its part, Uber might have suggested that it would use its
network of drivers and its scheduling software to recruit volunteer drivers
for a rescue mission.<em></em>
</p>
<p>Instead, we are told that Uber’s pricing surge <em>was</em> its expression
of concern. Uber’s way to address a human crisis is apparently by letting
the market govern human behavior, as if there were (in libertarian economist
Tyler Cowen’s phrase) “markets in everything” – including the lives of
a city’s beleaguered citizens (and its Uber drivers). <em></em>
</p>
<p>Where would this kind of market-driven practice leave poor or middle-income
citizens in a time of crisis? If they can’t afford the “surged” price,
apparently it would leave them squarely in the line of fire. And come to
think of it, why would Uber drivers value their lives so cheaply, unless
they’re underpaid? <em></em>
</p>
<p>One of the lessons of Sydney is this: Uber’s philosophy, whether consciously
expressed or not, is that life belongs to the highest bidder – and therefore,
by implication, the highest bidder’s life has the greatest value. Society,
on the other hand, may choose to believe that every life has equal value
– or that lifesaving services should be available at affordable prices. <em></em>
</p>
<p>If nothing else, the Sydney experience should prove once and for all that
there is no such thing as “the sharing economy.” Uber is a taxi company,
albeit an under-regulated one, and nothing more. It’s certainly not a “ride
sharing” service, where someone who happens to be going in the same direction
is willing to take along an extra passenger and split gas costs. A ride-sharing
service wouldn’t find itself “increasing fares to encourage more drivers”
to come into Sydney’s terrorized Central Business District. <em></em>
</p>
<p>A “sharing economy,” by definition, is lateral in structure. It is a peer-to-peer
economy. But Uber, as its name suggests, is hierarchical in structure.
It monitors and controls its drivers, demanding that they purchase services
from it while guiding their movements and determining their level of earnings.
And its pricing mechanisms impose unpredictable costs on its customers,
extracting greater amounts whenever the data suggests customers can be
compelled to pay them.<em></em>
</p>
<p>This is a top-down economy, not a “shared” one.<em></em>
</p>
<p>A number of Uber’s fans and supporters defended the company on the grounds
that its “surge prices,” including those seen during the Sydney crisis,
are determined by an algorithm. But an algorithm can be an ideological
statement, and is always a cultural artifact. As human creations, algorithms
reflect their creators. <em></em>
</p>
<p>Uber’s tweet during the Sydney crisis made it sound as if human intervention,
rather than algorithmic processes, caused prices to soar that day. But
it doesn’t really matter if that surge was manually or algorithmically
driven. Either way the prices were Uber’s doing – and its moral choice.<em></em>
</p>
<p>Uber has been strenuously defending its surge pricing in the wake of accusations
(apparently <a href="http://gothamist.com/2012/11/04/uber.php" target="_blank">justified</a>)
that the company enjoyed windfall profits during Hurricane Sandy. It has
now promised the state of New York that it will cap its surge prices (at
three times the highest rate on two non-emergency days). But if Uber has
its way, it will soon enjoy a monopolistic stranglehold on car service
rates in most major markets. And it has demonstrated its willingness to
ignore rules and regulations. That means<em> </em>predictable and affordable
taxi fares could become a thing of the past. <em></em>
</p>
<p>In practice, surge pricing could become a new, privatized form of taxation
on middle-class taxi customers.<em></em>
</p>
<p>Even without surge pricing, Uber and its supporters are hiding its full
costs. When middle-class workers are underpaid or deprived of benefits
and full working rights, as Uber’s <a href="http://www.businessinsider.com/uber-drivers-say-theyre-making-less-than-minimum-wage-2014-10" target="_blank">reportedly are</a>,
the entire middle-class economy suffers. Overall wages and benefits are
suppressed for the majority, while the wealthy few are made even richer.
The invisible costs of ventures like Uber are extracted over time, far
surpassing whatever short-term savings they may occasionally offer.<em></em>
</p>
<p>Like Walmart, Uber underpays its employees – many of its drivers <em>are</em> employees,
in everything but name – and then drains the social safety net to make
up the difference. While Uber preaches libertarianism, it practices a form
of corporate welfare. It’s reportedly <a href="http://www.washingtonpost.com/blogs/wonkblog/wp/2014/11/17/why-uber-loves-obamacare/" target="_blank">celebrating Obamacare</a>,
for example, since the Affordable Care Act allows it to avoid providing
health insurance to its workforce. But the ACA’s subsidies, together with
Uber’s often woefully insufficient wages, mean that the rest of us are
paying its tab instead. And the lack of income security among Uber’s drivers
creates another social cost for Americans – in lost tax revenue, and possibly
in increased use of social services. <em></em>
</p>
<p>The company’s war on regulation will also carry a social price. Uber and
its supporters don’t seem to understand that<em> </em>regulations exist
for a reason. It’s true that nobody likes excessive bureaucracy, but not
all regulations are excessive or onerous. And when they are, it’s a flaw
in execution rather than principle. <em></em>
</p>
<p>Regulations were created because they serve a social purpose, ensuring
the free and fair exchange of services and resources among all segments
of society. Some services, such as transportation, are of such importance
that the public has a vested interest in ensuring they will be readily
available at reasonably affordable prices. That’s not unreasonable for
taxi services, especially given the fact that they profit from publicly
maintained roads and bridges.<em></em>
</p>
<p>Uber has presented itself as a modernized, efficient alternative to government
oversight. But it’s an evasion of regulation, not its replacement. As
<a href="http://fusion.net/story/33680/the-inside-story-of-how-the-uber-portland-negotiations-broke-down/" target="_blank">Alexis Madrigal</a>reports, Uber has deliberately ignored city regulators
and used customer demand to force its model of inadequate self-governance
(my conclusion, not his) onto one city after another.<em></em>
</p>
<p>Uber presented itself as a refreshing alternative to the over-bureaucratized
world of urban transportation. But that’s a false choice. We can streamline
sclerotic city regulators, upgrade taxi fleets and even provide users with
fancy apps that make it easier to call a cab. The company’s binary presentation
– us, or City Hall – frames the debate in artificial terms.<em></em>
</p>
<p>Uber claims that its driver rating system is a more efficient way to monitor
drivers, but that’s an entirely unproven assumption. While taxi drivers
have been known to misbehave, the worldwide litany of complaints against
Uber drivers – for everything from dirty cars and <a href="http://consumerist.com/2014/07/30/uber-passenger-complains-of-spider-bite-in-filthy-car/" target="_blank">spider bites</a> to
<a href="http://www.forbes.com/sites/ellenhuet/2014/09/30/uber-driver-hammer-attack-liability/" target="_blank">assault with a hammer</a>, <a href="http://www.businessinsider.com/uber-nikki-williams-2014-12" target="_blank">fondling</a> and
<a href="http://www.businessinsider.com/an-uber-driver-allegedly-raped-a-female-passenger-in-boston-2014-12" target="_blank">rape</a>– suggest that Uber’s system may not work as well as old-fashioned
regulation. It’s certainly not noticeably superior.<em></em>
</p>
<p>In fact, <a href="http://www.huffingtonpost.com/2014/12/09/uber-california-lawsuit_n_6298206.html" target="_blank">prosecutors in San Francisco and Los Angeles</a> say
Uber has been lying to its customers about the level and quality of its
background checks. The company now promises it will do a better job at
screening drivers. But it <a href="http://consumerist.com/2014/12/18/uber-reportedly-revamping-security-wont-say-exactly-what-its-doing/" target="_blank">won’t tell us</a> what
measures its taking to improve its safety record, and it’s <a href="http://consumerist.com/2014/12/18/uber-reportedly-revamping-security-wont-say-exactly-what-its-doing/" target="_blank">fighting the kind of driver scrutiny</a> that
taxicab companies have been required to enforce for many decades. <em></em>
</p>
<p>Many reports suggest that beleaguered drivers don’t feel much better about
the company than victimized passengers do. They tell <a href="http://qz.com/299655/why-your-uber-driver-hates-uber/" target="_blank">horror stories</a> about
the company’s hiring and management practices. Uber <a href="http://www.salon.com/2014/09/03/uber_unrest_drivers_in_los_angeles_protest_the_slashing_of_rates/" target="_blank">unilaterally slashes drivers’ rates</a>,
while claiming they don’t need to unionize. (The <a href="http://www.fastcompany.com/3037371/the-teamsters-of-the-21st-century-how-uber-lyft-and-facebook-drivers-are-organizing" target="_blank">Teamsters</a> disagree.) <em></em>
</p>
<p>The company also pushes<a href="http://thinkprogress.org/economy/2014/11/06/3589715/uber-lending-investigation/" target="_blank"> sketchy, substandard loans</a> onto
its drivers – but hey, what could go wrong?<em></em>
</p>
<p>Uber has many libertarian defenders. And yet, it <a href="http://pando.com/2014/10/29/uber-prs-latest-trick-impersonating-its-drivers-and-trying-to-scam-journalists/" target="_blank">deceives the press</a> and
<a href="http://www.slate.com/blogs/the_slatest/2014/11/17/uber_exec_suggests_using_personal_info_against_journalists.html" target="_blank">threatens to spy on journalists</a>, <a href="http://money.cnn.com/2014/08/04/technology/uber-lyft/" target="_blank">lies to its own employees</a>,
keeps its practices a secret and routinely invades the privacy of civilians
– sometimes merely for entertainment. (It has a tool, with the Orwellian
name the “<a href="http://www.forbes.com/sites/kashmirhill/2014/10/03/god-view-uber-allegedly-stalked-users-for-party-goers-viewing-pleasure/" target="_blank">God View</a>,”
that it can use for monitoring customers’ personal movements.) <em></em>
</p>
<p>Aren’t those the kinds of things libertarians say they hate about <em>government</em>?<em></em>
</p>
<p>This isn’t a “gotcha” exercise. It matters. Uber is the poster child for
the pro-privatization, anti-regulatory ideology that ascribes magical powers
to technology and the private sector. It is deeply a political entity,
from its Nietzschean name to its recent hiring of White House veteran David
Plouffe. Uber is built around a relatively simple app (which relies on
government-created technology), but it’s not really a tech company. Above
all else Uber is an ideological campaign, a neoliberal project whose real
products are deregulation and the dismantling of the social contract.<em></em>
</p>
<p>Or maybe, as that tweeter in Sydney suggested, they’re just assholes.<em></em>
</p>
<p>Either way, it’s important that Uber’s worldview and business practices
not be allowed to “disrupt” our economy or our social fabric. People who
work hard deserve to make a decent living. Society at large deserves access
to safe and affordable transportation. And government, as the collective
expression of a democratic society, has a role to play in protecting its
citizens. <em></em>
</p>
<p>And then there’s the matter of our collective psyche. In her book “A Paradise
Built in Hell: The Extraordinary Communities that Arise in Disaster,” Rebecca
Solnit wrote of the purpose, meaning and deep satisfaction people find
when they pull together to help one another in the face of adversity. 
But in the world Uber seeks to create, those surges of the spirit would
be replaced by surge pricing.<em></em>
</p>
<p>You don’t need a “God view” to see what happens next. When heroism is
reduced to a transaction, the soul of a society is sold cheap. <em></em>
</p>
</div>
</DIV></article>

2513
resources/tests/readability/salon-1/source.html Normal file
View file

File diff suppressed because one or more lines are too long

179
resources/tests/readability/seattletimes-1/expected.html Normal file
View file

@ -0,0 +1,179 @@
<article><div itemprop="articleBody" id="readability-page-1">
<p>
From the deck of his 106-year-old halibut schooner, undergoing a seasonal overhaul at Fisherman’s Terminal in Seattle, skipper Wade Bassi has better insight than most into what’s happening at Amazon-owned Whole Foods Market, at least as pertains to the product he knows best.
</p>
<p>
While he doesn’t buy halibut much — he’s got a freezer full of it — Bassi, 43 years a fisherman, keeps an eye on how it’s handled and presented in the grocery stores and fish markets.
</p>
<p>
“When you look at nice halibut, I mean it is pure white,” he said. “And it is flaky-looking, and it is beautiful. It’s translucent. If you’ve got that in the fish market, people are going to buy it.”
</p>
<p>
A few days earlier, Whole Foods touted a rarely seen promotional price for halibut as part of its ongoing campaign to revise the grocery chain’s high-cost reputation while maintaining its image for quality and sustainability.
</p>
<p>
“Whole Foods is one of the better ones, to be honest with you,” Bassi said. “But you know, Whole Foods, whole paycheck. … They usually do charge way more for everything than anywhere else. Which really surprises me that they’re selling this for $16-something a pound, because they’re not making anything on it.”
</p>
<p>
Whole Foods’ halibut deal opens a window into Amazon’s grocery strategy as it seeks to combine the defining characteristics of each brand, leverage its juggernaut Prime membership program and take a larger share of the grocery business from competitors such as Walmart, Kroger and Costco.
</p>
<p>
It also draws a long line from a major Seattle industry with roots in the 19<sup>th</sup> century to the dominant economic force of the 21<sup>st</sup>.
</p>
<p>
Amazon bought Whole Foods in August 2017 for $13.7 billion, its largest acquisition and an aggressive move into the grocery business.
</p>
<p>
The combination of the two has been steady, said Tom Forte, who follows Amazon as a managing director at the D.A. Davidson brokerage. In a few more years, he said, “You won’t recognize the original Whole Foods.”
</p>
<p>
Within months of the acquisition, Forte said, Whole Foods was selling cheaper cage-free eggs and organic ground beef, prices it said were a result of the deal.
</p>
<p>
Then came the integration of Prime, Amazon’s $119-a-year shipping and media-subscription program, which Amazon founder Jeff Bezos said last year had surpassed 100 million members. In Whole Foods, Prime operates as a hybrid of the customer-loyalty discount programs offered by most grocers — in which consumers trade details of their purchasing habits for lower prices — and a paid membership like at Costco or Sam’s Club.
</p>
<p>
Whole Foods stores have been festooned with yellow and blue signs pointing out Prime member benefits, one of which was fresh halibut fillets priced at $16.99 a pound, albeit only for a week earlier this month.
</p>
<p>
“I was shocked to see that level,” said Tyler Besecker, president of Mercer Island-based Dana F. Besecker Company, the largest buyer of Pacific halibut. The price, which was matched at Kroger-owned QFC stores in the region last week, is “as low as I’ve ever seen.” (Besecker does not currently supply Whole Foods.)
</p>
<p>
Fresh halibut fillets routinely sell for $24 to $28 a pound, and often more.
</p>
<p>
He said there’s little if any room for a profit at the promotional price offered by Whole Foods and QFC. “They might be selling those at cost or as loss leaders just to get people into the stores,” Besecker said.
</p>
<p>
In the competitive grocery business, promotions like this happen all the time. The thinking is that shoppers will be attracted by the discount on a staple or a prestige item, and then fill their carts with other groceries sold at a profit.
</p>
<p>
A Whole Foods spokeswoman declined to comment on pricing. The temporary halibut discount is one of more than 300 such Prime promotions Whole Foods plans in the next few months. The company also said it was lowering prices across the store, its third such announcement since the acquisition.
</p>
<p>
At the seafood counter in the Whole Foods store on Westlake, surrounded by Amazon headquarters buildings, a sign advertised “First of the Season Fresh Alaskan Halibut” and sported the blue Marine Stewardship Council (MSC) Certified Sustainable Seafood label.
</p>
<p>
Whole Foods has been a pioneer in sustainable-seafood marketing, beginning in 1999 when it began to stock fish with the MSC label. In the mid-2000s, Pacific halibut fishermen sought the certification — a system of third-party audits that tracks seafood from catch to market — and Whole Foods was there from the beginning.
</p>
<p>
“They were the first ones to market the MSC halibut,” said Bob Alverson, head of the Fishing Vessel Owners Association, representing boats that catch halibut and black cod and a driver of the certification effort. “It turned into quite a marketing advantage. Whole Foods saw that early. They were focusing on sustainable, high-quality food products. They had quite a bit of foresight, I think, in that direction.”
</p>
<p>
The certification comes with added costs borne by the fishermen and buyers, and passed on to consumers. But it’s also an assurance “that people are watching out for the resource,” he said.
</p>
<p>
As it tries to convince people it has lower prices, Whole Foods has been very careful to maintain the reputation built on products like MSC-certified halibut.
</p>
<div>
<figure id="image-11519494">
<img data-ratio="1.5" data-caption="Amazon-owned Whole Foods touted a price cut on halibut as part of an announcement recently about lower prices on hundreds of items. (Ellen M. Banner / The Seattle Times)" alt="Amazon-owned Whole Foods touted a price cut on halibut as part of an announcement recently about lower prices on hundreds of items. (Ellen M. Banner / The Seattle Times)" src="https://static.seattletimes.com/wp-content/uploads/2019/04/120109-1020x680.jpg" srcset="https://static.seattletimes.com/wp-content/uploads/2019/04/120109-300x200.jpg 300w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-768x512.jpg 768w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-1024x683.jpg 1024w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-780x520.jpg 780w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-1020x680.jpg 1020w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-1560x1040.jpg 1560w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-375x250.jpg 375w" sizes="(max-width: 767px) calc(100vw - 20px), (max-width: 1019px) calc(100vw - 30px), (max-width: 1044px) calc(100vw - 60px), 970px" data-sizes="(max-width: 767px) calc(100vw - 20px), (max-width: 1019px) calc(100vw - 30px), (max-width: 1044px) calc(100vw - 60px), 970px" data-src="https://static.seattletimes.com/wp-content/uploads/2019/04/120109-1020x680.jpg" data-old-src="https://www.seattletimes.com/wp-content/themes/st_refresh/img/lazy-loading-14x9.png" data-srcset="https://static.seattletimes.com/wp-content/uploads/2019/04/120109-300x200.jpg 300w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-768x512.jpg 768w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-1024x683.jpg 1024w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-780x520.jpg 780w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-1020x680.jpg 1020w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-1560x1040.jpg 1560w, https://static.seattletimes.com/wp-content/uploads/2019/04/120109-375x250.jpg 375w">
<figcaption>
<span>Amazon-owned Whole Foods touted a price cut on halibut as part of an announcement recently about lower prices on hundreds of items. (Ellen M. Banner / The Seattle Times)</span>
</figcaption>
</figure>
</div>
<p>
<strong>Whole Foods future</strong>
</p>
<p>
New signs in stores appeared this month, spelling out the value proposition it’s trying to strike: “New lower prices. Same high standards,” reads one, against a background image of carrots.
</p>
<p>
At the same time, the company claims its new prices and Prime deals have saved customers “hundreds of millions of dollars” since the Amazon acquisition.
</p>
<p>
If prices are being lowered and the quality bar stays the same, something else has to give.
</p>
<p>
Analysts provided a few theories:
</p>
<p>
Amazon could be willing to accept losses or slimmer profits within Whole Foods, as it has done in other businesses, in an effort to expand its customer base.
</p>
<p>
“Do they take the profit from their non-retail efforts, which today is primarily cloud computing, and then reinvest those profits to take share in grocery?” Forte said.
</p>
<p>
That could eventually open up an avenue to growth as the rate of expansion slows in Amazon’s broader U.S. retail sales.
</p>
<p>
“That’s why they need grocery to work,” Forte said. “Grocery’s a very big category.”
</p>
<p>
Whole Foods could also <span data-st-annotation-ref="1792ba">press suppliers to reduce their prices</span>, essentially cutting their profit.
</p>
<p>
“The worry with that is that it puts the squeeze on the producer upstream,” said Ananth Iyer, a professor at Purdue University whose research includes sustainability in supply chains. If producers are squeezed too much, he noted, they may start to cut corners.
</p>
<p>
So far, there’s no evidence this is happening in halibut, where fishery practices have been carefully managed with a goal of sustainability for nearly a century. Also, Whole Foods does not yet have the scale as a buyer to dictate prices the way a company like Costco does.
</p>
<p>
Forte said that even if it did have such clout, this would be a risky strategy that would undermine the very attributes of the Whole Foods brand that make it most valuable.
</p>
<p>
Another theory is that Amazon could apply more of its technology and expertise in logistics to create supply-chain efficiencies that would maintain its profits while benefiting producers and consumers, particularly with perishable grocery products, Iyer said. This is part of the promise of the acquisition in the first place.
</p>
<p>
“That’s a powerful combination,” Forte said. “The sustainable, the organic, the healthier food — all those qualities of Whole Foods, with the supply-chain technology of Amazon. It plays to the strengths of both sides.”
</p>
<p>
Forte said he expects Amazon to continue its aggressive moves on grocery pricing at Whole Foods. But he wonders when the price cuts will be broader, particularly as Amazon competes with the likes of Walmart and Kroger for a bigger slice of U.S. food and beverage retail sales, which totaled $726 billion in 2017. Whole Foods said it has lowered prices on hundreds of items, with an emphasis on fresh produce.
</p>
<p>
Forte described his attempt after the acquisition to get the ingredients for Rice Krispies Treats at Whole Foods. It was perhaps doomed from the start: Whole Foods doesn’t carry Rice Krispies. But he found an organic brown rice puff cereal and organic marshmallows. They were “so wildly expensive that we didn’t finish the exercise. I took the kids to Walmart and bought the ingredients for a pittance,” Forte said.
</p>
<p>
That points to the bigger question of how Amazon plans to position Whole Foods for the long term in its expanding array of physical retail-grocery formats. It now has 11 automated Go convenience stores and is rumored to be planning a new, low-priced grocery chain of its own that may deploy the same cashierless checkout technology.
</p>
<p>
Meanwhile, Whole Foods is not opening any new 365 stores, a smaller, lower-priced version of the main brand highlighting the company’s private-label products. Whole Foods co-founder and CEO John Mackey said in an internal memo that the “price distinction between the two brands has become less relevant” as Whole Foods lowered its prices, Yahoo Finance <a href="https://finance.yahoo.com/news/amazonowned-whole-foods-scraps-smaller-365-store-expansion-203543290.html" target="_blank">reported</a> earlier this year.
</p>
<p>
Amazon, too, is finding success with a growing stable of private-label brands — it had more than 100 as of last July, according to Coresight Research, double the number in 2017. One of these, Solimo, sells generic versions of everything from K-Cup coffee pods to Epsom salts to garbage bags and racked up more than $6 million in sales in January alone, according to data analysis firm 1010data.
</p>
<p>
Of course, there’s nothing generic about a “fresh, sustainable wild-caught halibut fillet.”
</p>
<p>
<strong>Back on the schooner</strong>
</p>
<p>
The day after Easter, Bassi and his crew — three family members and an unrelated father-and-son team — loaded up the Polaris, one of four century-old wooden schooners still chasing halibut out of Seattle. (The broader Washington-based halibut fleet numbers about 100 vessels.) Bassi’s father fished on the Polaris, which Bassi co-owns with Rolfe McCartney. Bassi’s grandfather fished halibut back when schooners carried small dories out to the fishing grounds, which made the landing of a fish that can grow to 500 pounds all the more exciting.
</p>
<p>
The Polaris motored out of Fisherman’s Terminal and through the Ballard Locks to begin the three-day journey through the Inside Passage to Ketchikan, Alaska. There, they take on tons of ice and bait, herring for the black cod Bassi will target first, and later chum salmon, codfish or octopus for the halibut.
</p>
<p>
From a base in Kodiak, Alaska, the Polaris makes a series of trips, at sea for a week or longer at a time, to fish as far away as Attu Island at the far western edge of the Aleutian Islands chain. “It’s a big range that we fish,” Bassi said.
</p>
<p>
The Polaris will trail long lines of hooks, leaving them to soak for several hours before reeling them in. The fish are stunned, bled and dressed, and put on ice in the hold. It is this fishing method that contributes to the quality of the halibut and sustainability of the fishery, as it reduces by-catch — the inadvertent taking of other <span data-st-annotation-ref="53071e">species</span>.
</p>
<p>
They negotiate to sell the fish with four or five buyers, such as Besecker, at a price that fluctuates throughout the season. This is a peak time of year for halibut, with consumers seeking out fresh fish for Easter and Mother’s Day, Besecker said. Fishermen are typically paid between $5 and $6 a pound for halibut.
</p>
<p>
The Alaskan halibut fishery has its troubles — as nearly all fisheries do — but has been rationalized and managed successfully, particularly over the last quarter-century. Alverson, a commissioner on the Seattle-based International Pacific Halibut Commission, which has managed catch limits for U.S. and Canadian fishermen since 1924, described it as a stable but declining resource.
</p>
<p>
This year, the <a href="https://iphc.int/management/fisheries" target="_blank">halibut fishery</a> in the Northern Pacific and Bering Sea is <a href="https://iphc.int/data/landings-2019" target="_blank">capped at 29.4 million pounds</a>, with most of that allocated to commercial fishing and smaller amounts reserved for recreational and tribal fishing, as well as by-catch of other commercial fisheries.
</p>
<p>
Alverson summed up the journey from a wooden fishing boat in the Bering Sea to the fish counter of a grocery store owned by a company that has redefined modern buying and selling: “It’s Seattle old school meets Seattle new school with Amazon.”
</p>
</div></article>

1659
resources/tests/readability/seattletimes-1/source.html Normal file
View file

File diff suppressed because one or more lines are too long

18
resources/tests/readability/simplyfound-1/expected.html Normal file
View file

@ -0,0 +1,18 @@
<article><div id="readability-page-1">
<p>The Raspberry Pi Foundation started by a handful of volunteers in 2012 when they released the original Raspberry Pi 256MB Model B without knowing what to expect.  In a short four-year period they have grown to over sixty full-time employees and have shipped over <b>eight million</b> units to-date.  Raspberry Pi has achieved new heights by being shipped to the International Space Station for research and by being an affordable computing platforms used by teachers throughout the world.  "It has become the all-time best-selling computer in the UK".</p>
<p>Raspberry Pi 3 - A credit card sized PC that only costs $35 - Image: Raspberry Pi Foundation</p>
<p>Raspberry Pi Foundation is charity organization that pushes for a digital revolution with a mission to inspire kids to learn by creating computer-powered objects.  The foundation also helps teachers learn computing  skills through free training and readily available tutorials &amp; example code for creating cool things such as music.</p>
<p>Raspberry Pi in educations - Image: Raspberry Pi Foundation</p>
<p>In celebration of their 4th year anniversary, the foundation has released <b>Raspberry Pi 3</b> with the same price tag of<b> </b>$35 USD.  The 3rd revision features a <b>1.2GHz 64-bit quad-core</b> ARM CPU with integrated Bluetooth 4.1 and 802.11n wireless LAN chipsets.  The ARM Cortex-A53 CPU along with other architectural enhancements making it the fastest Raspberry Pi to-date.  The 3rd revision is reportedly about 50-60% times faster than its predecessor Raspberry Pi 2 and about 10 times faster then the original Raspberry PI.</p>
<p>Raspberry Pi - Various Usage</p>
<p>Raspberry Pi 3 is now available via many online resellers.  At this time, you should use a recent <b>32-bit </b>NOOBS or Raspbian image from their <a href="https://www.raspberrypi.org/downloads/" rel="nofollow" target="_blank">downloads</a> page with a promise of a switch to a 64-bit version only if further investigation proves that there is indeed some value in moving to 64-bit mode.</p>
</div></article>

426
resources/tests/readability/simplyfound-1/source.html Normal file
View file

@ -0,0 +1,426 @@
<!DOCTYPE html>
<html lang="en">
<!-- start of head -->
<head>
<link rel="dns-prefetch" href="//d34hb2g9mvfppu.cloudfront.net"/>
<script>
var media_url = '/';
</script>
<link rel="dns-prefetch" href="//d2ri69w25kq2xp.cloudfront.net"/>
<script>
var static_url = '/';
</script>
<link rel="dns-prefetch" href="//www.google-analytics.com"/>
<meta charset="utf-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge"/>
<meta property="og:url" content="https://simplyfound.com/article/eb9a5e137034/raspberry-pi-3-the-credit-card-sized-pc-that-cost-only-35-all-time-bestselling-computer-in-uk"/>
<meta property="og:title" content="Raspberry Pi 3 - The credit card sized PC that cost only $35 - All-time bestselling computer in UK"/>
<meta property="og:site_name" content="SIMPLYFOUND.COM | BY: JOE WEE"/>
<meta property="og:description" content="The Raspberry Pi Foundation started by a handful of volunteers in 2012 when they released the original Raspberry Pi 256MB Model B without knowing what to expect. In a short four-year period they have grown to over sixty full-time employees and ha..."/>
<meta property="fb:app_id" content="430934063772847"/>
<meta name="twitter:site" content="SIMPLYFOUND.COM | BY JOE WEE"/>
<meta name="twitter:title" content="Raspberry Pi 3 - The credit card sized PC that cost only $35 - All-time bestselling computer in UK"/>
<meta name="twitter:description" content="The Raspberry Pi Foundation started by a handful of volunteers in 2012 when they released the original Raspberry Pi 256MB Model B without knowing what to expect. In a short four-year period they have grown to over sixty full-time employees and ha..."/>
<meta property="og:image" content="https://d34hb2g9mvfppu.cloudfront.net/m/images/cache/images/2016/02/29/apcnews2012raspberry_pi_logo_mainimage8_jpg8_322_27630a8388eb_lg.jpg"/>
<meta name="twitter:image" content="https://d34hb2g9mvfppu.cloudfront.net/m/images/cache/images/2016/02/29/apcnews2012raspberry_pi_logo_mainimage8_jpg8_322_27630a8388eb_lg.jpg"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no"/>
<meta name="apple-mobile-web-app-capable" content="yes"/>
<title>Raspberry Pi 3 - The credit card sized PC that cost only $35 - All-time bestselling computer in UK - SimplyFound</title>
<meta name="description" content="The Raspberry Pi Foundation started by a handful of volunteers in 2012 when they released the original Raspberry Pi 256MB Model B without knowing what to expect. In a short four-year period they have grown to over sixty full-time employees and ha..."/>
<meta name="author" content=""/>
<link rel="shortcut icon" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/favicon.351a323a1b51.ico" type="image/x-icon"/>
<link rel="apple-touch-icon" sizes="57x57" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-57x57.5cb9f8349ece.png"/>
<link rel="apple-touch-icon" sizes="60x60" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-60x60.4f08bf97ade1.png"/>
<link rel="apple-touch-icon" sizes="72x72" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-72x72.4d2746f1e06b.png"/>
<link rel="apple-touch-icon" sizes="76x76" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-76x76.9b7be849c186.png"/>
<link rel="apple-touch-icon" sizes="114x114" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-114x114.a1012e6d710d.png"/>
<link rel="apple-touch-icon" sizes="120x120" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-120x120.e57fe7c36b78.png"/>
<link rel="apple-touch-icon" sizes="144x144" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-144x144.b8ecc451ebd4.png"/>
<link rel="apple-touch-icon" sizes="152x152" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-152x152.dc67c8151e84.png"/>
<link rel="apple-touch-icon" sizes="180x180" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/apple-icon-180x180.7cf140b4bc83.png"/>
<link rel="icon" type="image/png" sizes="192x192" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/android-icon-192x192.8de435c48446.png"/>
<link rel="icon" type="image/png" sizes="32x32" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/favicon-32x32.6fc201c7c810.png"/>
<link rel="icon" type="image/png" sizes="96x96" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/favicon-96x96.973e422a0ea5.png"/>
<link rel="icon" type="image/png" sizes="16x16" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/favicon-16x16.3e1ed5bfe029.png"/>
<link rel="manifest" href="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/manifest.b58fcfa7628c.json"/>
<meta name="msapplication-TileColor" content="#ffffff"/>
<meta name="msapplication-TileImage" content="https://d2ri69w25kq2xp.cloudfront.net/s/img/favicons/ms-icon-144x144.b8ecc451ebd4.png"/>
<meta name="theme-color" content="#ffffff"/>
<link href="https://d2ri69w25kq2xp.cloudfront.net/s/css/vendor/bootstrap/css/bootstrap.min.12d017d85b55.css" rel="stylesheet"/>
<link href="https://d2ri69w25kq2xp.cloudfront.net/s/css/vendor/font-awesome/css/font-awesome.min.4fbd15cb6047.css" rel="stylesheet"/>
<link href="https://d2ri69w25kq2xp.cloudfront.net/s/css/vendor/toastr.min.fba9df3af906.css" rel="stylesheet"/>
<link href="https://d2ri69w25kq2xp.cloudfront.net/s/css/bs-style.c1681681331f.css" rel="stylesheet"/>
<style type="text/css">
.input-ajax-wheel {
background: url("https://d2ri69w25kq2xp.cloudfront.net/s/img/misc/ajax_loader_red.514ee5681a2a.gif") 8px no-repeat;
background-position-x: 99%;
display: none;
}
.new-divider {
background: transparent url("https://d2ri69w25kq2xp.cloudfront.net/s/img/misc/bg-1px-grey.fd1962149515.gif") repeat-x left 9px;
}
</style>
</head>
<!-- end of head -->
<!-- start of body -->
<body>
<div id="main-navbar" class="top-nav">
<nav class="navbar navbar-inverse navbar-fixed-top">
<div class="container-fluid">
<div class="navbar-header">
<div class="pull-right">
<div class="search-menu-item pull-left">
<ul class="nav navbar-nav">
<li class=""><a href="javascript:void(0)" rel="nofollow,noindex" class="text-search"><i class="fa fa-search"></i></a></li>
</ul>
</div>
<button class="navbar-toggle collapsed" data-toggle="collapse" data-target="#collapsable-top-nav"> <span class="menu-caret"><i class="fa fa-navicon"></i><i class="fa fa-caret-down"></i></span> </button>
</div> <a class="navbar-brand" href="/" title="Home">SimplyFound</a> </div>
<div class="collapse navbar-collapse" id="collapsable-top-nav">
<ul class="nav navbar-nav navbar-left">
<li class="">
<a href="/featured" class=""> <span> Featured</span> <i class="fa fa-check"></i> </a>
</li>
<li class="">
<a href="/popular" class=""> <span> Popular</span> <i class="fa fa-star"></i> </a>
</li>
<li class="">
<a href="/latest" class=""> <span> Latest</span> <i class="fa fa-clock-o"></i> </a>
</li>
</ul>
<ul class="nav navbar-nav navbar-right">
<li class="">
<a href="/account/user/login" class=""> <span> Login</span> </a>
</li>
</ul>
</div>
</div>
</nav>
</div>
<div id="nav-spacer"></div>
<div id="text-search-modal" class="col-lg-12 modal fade">
<div class="modal-dialog">
<div class="modal-content">
<div class="modal-body">
<form id="text-search-form" class="navbar-form" method="get" action="">
<div class="input-group"> <span id="text-search-clear" class="input-group-btn">
<button class="btn btn-default" type="button"><i class="fa fa-times"></i></button>
</span>
<input type="text" id="text-search-input" placeholder="Search..." class="form-control" data-referrer="latest"/> <span class="input-group-btn">
<button class="btn btn-primary text-search-submit-button" type="button">Go</button>
</span> </div>
</form>
</div>
</div>
</div>
</div>
<div class="article-view-box">
<div class="col-md-12 article-content">
<div class="col-md-12 unfloat article center-block">
<div class="article-fullview center-block">
<div class="hidden">137</div>
<style>
.top_add {
width: 320px;
height: 50px;
margin-bottom: 10px;
}
@media(min-width: 500px) {
.top_add {
width: 100%;
height: 67px;
}
}
@media(min-width: 500px) {
.top_add {
width: 100%;
height: 67px;
}
}
</style>
<center> <ins class="adsbygoogle top_add" style="display:inline-block;" data-ad-client="ca-pub-9052696005983480" data-ad-slot="3867410294" data-ad-format="auto">
</ins> </center>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
<div class="card-box-content box-shadow-4dp">
<div class="card-box-header">
<h1 class="card-box-title">
Raspberry Pi 3 - The credit card sized PC that cost only $35 - All-time bestselling computer in UK
</h1>
<div class="article-author"> <i class="fa fa-user"></i> <span class="rotate-wrapper">
<a href="/latest?author=joewee" rel="index,follow"><span>Joe Wee</span></a>&nbsp; <i class="fa fa-thumb-tack text-success fa-rotate-45"></i> </span> &nbsp;&nbsp;<i class="fa fa-clock-o"></i> <span>
Monday, February 29, 2016 @ 11:10 PM UTC
</span> </div>
<div class="article-category-tags">
<a href="/latest?category=consumer-electronics" rel="noindex,follow" type="button" class="btn btn-default rotate-wrapper "> <span class="category-toggle-button">Consumer Electronics</span> <i class="fa fa-thumb-tack text-success fa-rotate-45"></i> </a>
<a href="/latest?category=technology" rel="noindex,follow" type="button" class="btn btn-default rotate-wrapper "> <span class="category-toggle-button">Technology</span> <i class="fa fa-thumb-tack text-success fa-rotate-45"></i> </a>
<a href="/latest?tag=pc" rel="noindex,follow" type="button" class="btn btn-default rotate-wrapper "> <span class="tag-toggle-button">PC</span> <i class="fa fa-thumb-tack text-success fa-rotate-45"></i> </a>
<a href="/latest?tag=raspberry-pi" rel="noindex,follow" type="button" class="btn btn-default rotate-wrapper "> <span class="tag-toggle-button">Raspberry Pi</span> <i class="fa fa-thumb-tack text-success fa-rotate-45"></i> </a>
<a href="/latest?tag=computer" rel="noindex,follow" type="button" class="btn btn-default rotate-wrapper "> <span class="tag-toggle-button">Computer</span> <i class="fa fa-thumb-tack text-success fa-rotate-45"></i> </a>
<a href="/latest?tag=raspberry-pi-3" rel="noindex,follow" type="button" class="btn btn-default rotate-wrapper "> <span class="tag-toggle-button">Raspberry Pi 3</span> <i class="fa fa-thumb-tack text-success fa-rotate-45"></i> </a>
</div>
</div>
<div class="card-box-body">
<div class="text-content-box">
<p>The Raspberry Pi Foundation started by a handful of volunteers in 2012 when they released the original Raspberry Pi 256MB Model B without knowing what to expect. &nbsp;In a short four-year period they have grown to over sixty full-time&nbsp;employees&nbsp;and have shipped over <b>eight million</b> units to-date. &nbsp;Raspberry Pi has achieved new heights by being shipped to the&nbsp;International&nbsp;Space Station for research and by being an affordable computing platforms used by teachers throughout the world. &nbsp;"It has become the all-time best-selling computer in the UK".</p>
</div>
<div class="spacer"></div>
<div class="media-container ">
<div class="image-wrapper">
<div class="image"> <img src="https://d34hb2g9mvfppu.cloudfront.net/m/images/cache/images/2016/02/29/apcnews2012raspberry_pi_logo_mainimage8_jpg8_322_27630a8388eb_lg.jpg" class="bottom-aligned"/> </div>
</div>
</div>
<div class="media-caption">Raspberry Pi 3 - A credit card sized PC that only costs $35 - Image: Raspberry Pi Foundation</div>
<div class="spacer"></div>
<div class="text-content-box">
<p>Raspberry Pi Foundation is charity organization that pushes for a digital revolution with a mission to inspire kids to learn by&nbsp;creating computer-powered objects. &nbsp;The foundation also helps teachers learn computing &nbsp;skills through free training and readily available tutorials &amp; example code for creating cool things such as music.</p>
</div>
<div class="spacer"></div>
<div class="media-container ">
<div class="image-wrapper">
<div class="image"> <img src="https://d34hb2g9mvfppu.cloudfront.net/m/images/cache/images/2016/02/29/teachers_classroom_guide_324_a221bf31d64c_lg.png" class="bottom-aligned"/> </div>
</div>
</div>
<div class="media-caption">Raspberry Pi in educations - Image: Raspberry Pi Foundation</div>
<div class="spacer"></div>
<div class="text-content-box">
<p>In celebration of their 4th year&nbsp;anniversary, the foundation has released&nbsp;<b>Raspberry Pi 3</b> with the same price tag of<b>&nbsp;</b>$35 USD. &nbsp;The 3rd revision features a <b>1.2GHz 64-bit quad-core</b>&nbsp;ARM CPU with integrated Bluetooth 4.1 and 802.11n wireless LAN chipsets. &nbsp;The ARM Cortex-A53 CPU along with other architectural enhancements making it the fastest Raspberry Pi to-date. &nbsp;The 3rd revision is reportedly about 50-60% times faster than its predecessor Raspberry Pi 2 and about 10 times faster then the original Raspberry PI.</p>
</div>
<div class="spacer"></div>
<div class="media-container ">
<div id="snippet-326-image-carousel" class="carousel slide image-carousel" data-ride="carousel" data-interval="false">
<ol class="carousel-indicators">
<li data-target="#snippet-326-image-carousel" data-slide-to="0" class=" active "></li>
<li data-target="#snippet-326-image-carousel" data-slide-to="1" class=""></li>
<li data-target="#snippet-326-image-carousel" data-slide-to="2" class=""></li>
</ol>
<div class="carousel-inner">
<div class="item active ">
<div class="image-wrapper">
<div class="image"> <img src="https://d34hb2g9mvfppu.cloudfront.net/m/images/cache/images/2016/02/29/handj_326_eb8b50597a3f_lg.jpg" alt="Image 1" class="bottom-aligned"/> </div>
</div>
</div>
<div class="item ">
<div class="image-wrapper">
<div class="image"> <img src="https://d34hb2g9mvfppu.cloudfront.net/m/images/cache/images/2016/02/29/images_326_b1f81e087284_lg.jpeg" alt="Image 2" class="bottom-aligned"/> </div>
</div>
</div>
<div class="item ">
<div class="image-wrapper">
<div class="image"> <img src="https://d34hb2g9mvfppu.cloudfront.net/m/images/cache/images/2016/02/29/images_2_326_c32fa7688f70_lg.jpeg" alt="Image 3" class="bottom-aligned"/> </div>
</div>
</div>
</div>
<a href="#snippet-326-image-carousel" data-slide="prev" class="carousel-control left">
<div class="overlay-left-button" style="background: url('https://d2ri69w25kq2xp.cloudfront.net/s/img/misc/carousel-arrow-stripes.35420672cdd8.png') no-repeat;"></div>
</a>
<a href="#snippet-326-image-carousel" data-slide="next" class="carousel-control right">
<div class="overlay-right-button" style="background: url('https://d2ri69w25kq2xp.cloudfront.net/s/img/misc/carousel-arrow-stripes.35420672cdd8.png') no-repeat;"></div>
</a>
</div>
</div>
<div class="media-caption">Raspberry Pi - Various Usage</div>
<div class="spacer"></div>
<div class="text-content-box">
<p>Raspberry Pi 3 is now available via many online resellers. &nbsp;At this time, you should use a recent <b>32-bit </b>NOOBS or Raspbian image from their&nbsp;<a href="https://www.raspberrypi.org/downloads/" rel="nofollow" target="_blank">downloads</a> page with a promise of a switch to a 64-bit version only if further investigation proves that there is indeed some value in moving to 64-bit mode.</p>
</div>
</div>
<div class="article-view-action text-center">
<div class="btn btn-default viewcount bold"> <i class="fa fa-eye"></i><span class="action-button-text">137 </span> </div>
<a href="/account/user/login?next=/article/eb9a5e137034/raspberry-pi-3-the-credit-card-sized-pc-that-cost-only-35-all-time-bestselling-computer-in-uk" rel="noindex" class="btn btn-default actionable bold confirm" title="Login to like?" placement="top"> <i class="fa fa-heart-o"></i> <span class="action-button-text trac-togglable-value">1</span> </a>
<a href="#" rel="noindex" class="btn btn-default actionable bold" data-toggle="modal" data-target="#social-sharing-modal-article-66"> <i class="fa fa-share-alt"></i><span class="action-button-text">Share</span> </a>
<a href="#" rel="noindex" class="btn btn-default actionable article-history bold hidden"> <i class="fa fa-arrow-left"></i><span class="action-button-text">Back</span> </a>
</div>
</div>
</div>
<div id="social-sharing-modal-article-66" class="col-lg-12 modal fade social-share-box">
<div class="modal-dialog">
<div class="modal-content">
<div class="modal-header">
<button type="button" class="close" data-dismiss="modal"><i class="fa fa-times"></i></button>
<h4 class="modal-title bold">Like It, Share It!</h4> </div>
<div class="modal-body">
<a href="https://www.facebook.com/sharer/sharer.php?u=https%3A//simplyfound.com/article/eb9a5e137034" rel="nofollow" class="social-share-link"> <span class="fa-stack fa-3x" rel="tooltip" data-placement="top" data-original-title="Facebook">
<i class="fa fa-circle fa-stack-2x facebook-bg"></i>
<i class="fa fa-facebook-f fa-stack-1x social-fg"></i>
</span> </a>
<a href="https://twitter.com/intent/tweet?url=https%3A//simplyfound.com/article/eb9a5e137034&amp;text=Raspberry%20Pi%203%20-%20The%20credit%20card%20sized%20PC%20that%20cost%20only%20%2435%20-%20All-time%20bestselling%20computer%20in%20UK" rel="nofollow" class="social-share-link"> <span class="fa-stack fa-3x" rel="tooltip" data-placement="top" data-original-title="Twitter">
<i class="fa fa-circle fa-stack-2x twitter-bg"></i>
<i class="fa fa-twitter fa-stack-1x social-fg"></i>
</span> </a>
<a href="https://plus.google.com/share?url=https%3A//simplyfound.com/article/eb9a5e137034" rel="nofollow" class="social-share-link"> <span class="fa-stack fa-3x" rel="tooltip" data-placement="top" data-original-title="Google">
<i class="fa fa-circle fa-stack-2x google-bg"></i>
<i class="fa fa-google-plus fa-stack-1x social-fg"></i>
</span> </a>
<a href="https://www.linkedin.com/shareArticle?mini=true&amp;url=https%3A//simplyfound.com/article/eb9a5e137034&amp;title=Raspberry%20Pi%203%20-%20The%20credit%20card%20sized%20PC%20that%20cost%20only%20%2435%20-%20All-time%20bestselling%20computer%20in%20UK&amp;source=SimplyFound" rel="nofollow" class="social-share-link"> <span class="fa-stack fa-3x" rel="tooltip" data-placement="top" data-original-title="Linkedin">
<i class="fa fa-circle fa-stack-2x linkedin-bg"></i>
<i class="fa fa-linkedin fa-stack-1x social-fg"></i>
</span> </a>
<a href="https://www.reddit.com/submit?url=https%3A//simplyfound.com/article/eb9a5e137034&amp;title=Raspberry%20Pi%203%20-%20The%20credit%20card%20sized%20PC%20that%20cost%20only%20%2435%20-%20All-time%20bestselling%20computer%20in%20UK" rel="nofollow" class="social-share-link"> <span class="fa-stack fa-3x" rel="tooltip" data-placement="top" data-original-title="Reddit">
<i class="fa fa-circle fa-stack-2x reddit-bg"></i>
<i class="fa fa-reddit-alien fa-stack-1x social-fg"></i>
</span> </a>
<a href="https://pinterest.com/pin/create/button/?url=https%3A//simplyfound.com/article/eb9a5e137034&amp;description=Raspberry%20Pi%203%20-%20The%20credit%20card%20sized%20PC%20that%20cost%20only%20%2435%20-%20All-time%20bestselling%20computer%20in%20UK&amp;media=https://d34hb2g9mvfppu.cloudfront.net/m/images/cache/images/2016/02/29/apcnews2012raspberry_pi_logo_mainimage8_jpg8_322_7490af98d726_sm.jpg" rel="nofollow" class="social-share-link"> <span class="fa-stack fa-3x" rel="tooltip" data-placement="top" data-original-title="Pinterest">
<i class="fa fa-circle fa-stack-2x pinterest-bg"></i>
<i class="fa fa-pinterest fa-stack-1x social-fg"></i>
</span> </a>
<a href="mailto:?subject=Raspberry Pi 3 - The credit card sized PC that cost only $35 - All-time bestselling computer in UK&amp;body=I am sharing this article with you. Check it out! %0D%0A%0D%0A https%3A//simplyfound.com/article/eb9a5e137034 %0D%0A%0D%0A ------------- %0D%0A Thursday, March 17, 2016 @ 6:38 AM" rel="nofollow" class="social-share-link"> <span class="fa-stack fa-3x" rel="tooltip" data-placement="top" data-original-title="Email">
<i class="fa fa-circle fa-stack-2x email-bg"></i>
<i class="fa fa-envelope-o fa-stack-1x social-fg"></i>
</span> </a>
</div>
</div>
</div>
</div>
<div id="become-an-apporved-author" class="col-lg-12 modal fade">
<div class="modal-dialog">
<div class="modal-content">
<div class="modal-header">
<button type="button" class="close" data-dismiss="modal"><i class="fa fa-times"></i></button>
<h4 class="modal-title bold">Become an approved author!</h4> </div>
<div class="modal-body"> <i class="fa fa-hand-o-right red"></i>&nbsp;&nbsp;You account is not approved yet.
<br/>
<br/> To become an approved author, you must have minimum of two articles in your account.
<br/>
<br/> The articles must be complete and ready to be published.
<br/>
<br/> The articles must be unique and contain no duplicate contents from other websites, books and/or magazines.
<br/>
<br/> You think you are ready? If so, request to become an approved author.
<br/>
<br/> <i class="fa fa-hand-o-right red"></i>&nbsp;&nbsp;[
<a class="info" href="/approval/request"> <span>Send Your Request</span> </a> ] </div>
</div>
</div>
</div>
<style>
.bottom_ad {
width: 320px;
height: 50px;
margin-top: 10px;
}
@media(min-width: 500px) {
.bottom_ad {
width: 100%;
height: 67px;
}
}
@media(min-width: 500px) {
.bottom_ad {
width: 100%;
height: 67px;
}
}
</style>
<center> <ins class="adsbygoogle bottom_ad" style="display:inline-block;" data-ad-client="ca-pub-9052696005983480" data-ad-slot="3867410294" data-ad-format="auto">
</ins> </center>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
</div>
</div>
</div>
<div class="clearfix"></div>
<div class="main-footer">
<nav class="bg-silver">
<div class="col-md-3 footer-logo">
<a href="/">
<div class="logo-inverse-large"></div>
</a>
</div>
<div class="col-md-3 footer-column">
<h3 class="text-headline text-white">SEARCH</h3>
<ul class="text-body">
<li>
<a href="/featured" class="" rel="nofollow,noindex"> <i class="fa fa-check"></i>&nbsp;&nbsp;<span> Featured</span> </a>
</li>
<li>
<a href="/popular" class="" rel="nofollow,noindex"> <i class="fa fa-star"></i>&nbsp;&nbsp;<span> Popular</span> </a>
</li>
<li>
<a href="/latest" class="" rel="nofollow,noindex"> <i class="fa fa-clock-o"></i>&nbsp;&nbsp;<span> Latest</span> </a>
</li>
</ul>
</div>
<div class="col-md-3 footer-column">
<h3 class="text-headline text-white">SOCIAL</h3>
<ul class="text-body">
<li>
<a href="https://www.facebook.com/SimplyFound-1500754540232504" class="" rel="nofollow,noindex"> <i class="fa fa-facebook-square"></i>&nbsp;&nbsp;<span> Facebook</span> </a>
</li>
<li>
<a href="https://twitter.com/simplyfound" class="" rel="nofollow,noindex"> <i class="fa fa-twitter-square"></i>&nbsp;&nbsp;<span> Twitter</span> </a>
</li>
<li>
<a href="/blog" class="" rel="nofollow,noindex"> <i class="fa fa-rss-square"></i>&nbsp;&nbsp;<span> Blog</span> </a>
</li>
</ul>
</div>
<div class="col-md-3 footer-column">
<h3 class="text-headline text-white">COMPANY</h3>
<ul class="text-body">
<li>
<a href="/site/terms-conditions" class="" rel="nofollow,noindex"> <i class="fa fa-file-text-o"></i>&nbsp;&nbsp;<span> Terms &amp; Conditions</span> </a>
</li>
<li>
<a href="/site/privacy-policy" class="" rel="nofollow,noindex"> <i class="fa fa-file-text-o"></i>&nbsp;&nbsp;<span> Privacy Policy</span> </a>
</li>
<li>
<a href="/contact/" class="" rel="nofollow,noindex"> <i class="fa fa-envelope-o"></i>&nbsp;&nbsp;<span> Contact Us</span> </a>
</li>
</ul>
</div>
</nav>
<div class="credit bg-steel"> <small class="text-caption">
<a href="/" class="unfloat">
© 2016<strong> &nbsp; Simplyfound.com</strong>
</a>
</small> </div>
</div>
<script src="https://d2ri69w25kq2xp.cloudfront.net/s/js/vendor/jquery.min.f9c7afd05729.js"></script>
<script src="https://d2ri69w25kq2xp.cloudfront.net/s/js/vendor/bootstrap.min.c5b5b2fa19bd.js"></script>
<script src="https://d2ri69w25kq2xp.cloudfront.net/s/js/vendor/confirmation.min.e22246a9e3ea.js"></script>
<script src="https://d2ri69w25kq2xp.cloudfront.net/s/js/vendor/autohidingnavbar.min.8df1fc2b1810.js"></script>
<script src="https://d2ri69w25kq2xp.cloudfront.net/s/js/vendor/toastr.min.fc293118b38b.js"></script>
<script src="https://d2ri69w25kq2xp.cloudfront.net/s/js/utils.415bd0240799.js"></script>
<script src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$('[rel=tooltip]').tooltip();
$("[rel='popover']").popover();
$(".top-nav .navbar-fixed-top").autoHidingNavbar();
});
</script>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-40482276-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script');
ga.type = 'text/javascript';
ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
})();
</script>
<form id="csrf_post_form" action="" enctype="multipart/form-data" method="post" accept-charset="utf-8">
<input type="hidden" name="csrfmiddlewaretoken" value="CT1Z3UzIHniUTpMJGzlkYYIJg68vGUaf"/> </form>
<div id="message_box"> </div>
</body>
<!-- end of body -->
</html>

34
resources/tests/readability/social-buttons/expected.html Normal file
View file

@ -0,0 +1,34 @@
<article><DIV id="readability-page-1"><article>
<h2>Lorem ipsum dolor</h2>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
</article></DIV></article>

54
resources/tests/readability/social-buttons/source.html Normal file
View file

@ -0,0 +1,54 @@
<!DOCTYPE html>
<html>
<head>
<title>Share buttons removal test</title>
</head>
<body>
<article class="main-content">
<h1>Lorem ipsum dolor</h1>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<div class="sharedaddy sd-block sd-like" id="like-post-wrapper-70135762-15074-552441e572442"
data-src="//widgets.wp.com/likes/#blog_id=70135762&amp;post_id=15074&amp;origin=longreadsblog.wordpress.com&amp;obj_id=70135762-15074-552441e572442"
data-name="like-post-frame-70135762-15074-552441e572442">
<h3 class="sd-title">Like this:</h3>
<div class="likes-widget-placeholder post-likes-widget-placeholder" style="height:55px">
<span class="button">
<span>Like</span>
</span>
<span class="loading">Loading...</span>
</div>
<span class="sd-text-color"></span>
<a class="sd-link-color"></a>
</div>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
</article>
</body>
</html>

7
src/constants.rs
View file

@ -60,7 +60,12 @@ pub static POSITIVE: Lazy<Regex> =
pub static NEGATIVE: Lazy<Regex> = Lazy::new(|| { pub static NEGATIVE: Lazy<Regex> = Lazy::new(|| {
RegexBuilder::new(r#"-ad-|hidden|^hid$| hid$| hid |^hid |banner|combx|comment|com-|contact|foot|footer|footnote|gdpr|masthead|media|meta|outbrain|promo|related|scroll|share|shoutbox|sidebar|skyscraper|sponsor|shopping|tags|tool|widget"#).case_insensitive(true).build().expect("NEGATIVE regex") RegexBuilder::new(r#"-ad-|hidden|^hid$| hid$| hid |^hid |banner|combx|comment|com-|contact|foot|footer|footnote|gdpr|masthead|media|meta|outbrain|promo|related|scroll|share|shoutbox|sidebar|skyscraper|sponsor|shopping|tags|tool|widget"#).case_insensitive(true).build().expect("NEGATIVE regex")
}); });
pub static SHARE_ELEMENTS: Lazy<Regex> = Lazy::new(|| {
RegexBuilder::new(r#"(\b|_)(share|sharedaddy)(\b|_)"#)
.case_insensitive(true)
.build()
.expect("SHARE_ELEMENTS regex")
});
pub static TITLE_SEPARATOR: Lazy<Regex> = pub static TITLE_SEPARATOR: Lazy<Regex> =
Lazy::new(|| Regex::new(r#" [-|—\\/>»] "#).expect("TITLE_SEPARATOR regex")); Lazy::new(|| Regex::new(r#" [-|—\\/>»] "#).expect("TITLE_SEPARATOR regex"));
pub static TITLE_CUT_END: Lazy<Regex> = Lazy::new(|| { pub static TITLE_CUT_END: Lazy<Regex> = Lazy::new(|| {

21
src/full_text_parser/mod.rs
View file

@ -975,6 +975,7 @@ impl FullTextParser {
Util::clean_conditionally(node, "ul"); Util::clean_conditionally(node, "ul");
Util::clean_conditionally(node, "div"); Util::clean_conditionally(node, "div");
Self::remove_share_elements(node);
Self::clean_attributes(node)?; Self::clean_attributes(node)?;
Self::remove_single_cell_tables(node); Self::remove_single_cell_tables(node);
Self::remove_extra_p_and_div(node); Self::remove_extra_p_and_div(node);
@ -1042,6 +1043,26 @@ impl FullTextParser {
} }
} }
fn remove_share_elements(root: &mut Node) {
let mut node_iter = Some(root.clone());
while let Some(mut node) = node_iter {
let match_string = format!(
"{} {}",
node.get_attribute("class").unwrap_or_default(),
node.get_attribute("id").unwrap_or_default()
);
if constants::SHARE_ELEMENTS.is_match(&match_string)
&& node.get_content().len() < constants::DEFAULT_CHAR_THRESHOLD
{
node_iter = Util::remove_and_next(&mut node);
} else {
node_iter = Util::next_node(&node, false);
}
}
}
fn clean_attributes(root: &mut Node) -> Result<(), FullTextParserError> { fn clean_attributes(root: &mut Node) -> Result<(), FullTextParserError> {
let mut node_iter = Some(root.clone()); let mut node_iter = Some(root.clone());

6
src/full_text_parser/readability/mod.rs
View file

@ -705,7 +705,11 @@ impl Readability {
0 0
}; };
let score = score + class_weight; let score = score + class_weight;
log::debug!("initialize node {} {}: {score}", node.get_name(), node.get_attribute("class").unwrap_or_default()); log::debug!(
"initialize node {} {}: {score}",
node.get_name(),
node.get_attribute("class").unwrap_or_default()
);
Self::set_content_score(node, score as f64)?; Self::set_content_score(node, score as f64)?;
Ok(()) Ok(())
} }

20
src/full_text_parser/readability/tests.rs
View file

@ -438,6 +438,26 @@ async fn replace_font_tags() {
run_test("replace-font-tags").await run_test("replace-font-tags").await
} }
#[tokio::test]
async fn salon_1() {
run_test("salon-1").await
}
#[tokio::test]
async fn seattletimes_1() {
run_test("seattletimes-1").await
}
#[tokio::test]
async fn simplyfound_1() {
run_test("simplyfound-1").await
}
#[tokio::test]
async fn social_buttons() {
run_test("social-buttons").await
}
#[tokio::test] #[tokio::test]
async fn webmd_1() { async fn webmd_1() {
run_test("webmd-1").await run_test("webmd-1").await