The textracto web data extraction API, is a simple API allowing you to extract content from websites. Responses are available in JSON. Check out the live demo.

Overview

PathParametersRequiredtypeDescription Example
/keytrueHashYour API key.97e491814155ffb3f6a8dc3efzz853830d914136
/urltrueStringThe full URL of the page you want to extracthttp://www.bbc.com/news/election-us-2016-37104688
/contentsfalseStringSpecify the kind of content you want to extract. See table below for further details.thm
/cachefalseInteger'1' if the API should use cached content faster, '0' if not.1
/jsfalseInteger'1' if Javascript should be executed (slow!)1

The ‘contents’ parameter

The ‘contents’ parameter allows you to specify the kind of content you want to get returned from the API. By default, without setting the ‘contents’ parameter, the API will return: the main textual content, meta data and links and images which were found on the page.

ExampleDescription
&contents=mReturns metadata.
&contents=tReturns the main textual content.
&contents=hReturns links and images that were found the specified url.
&contents=dReturns schema.org microdata
&contents=aReturns all text that were found on the page.
&contents=athdYou can also combine the parameters, the order isn't import. This example would return: all text, the main text, links and images, schema.org microdata.

Examples

You can use POST or GET.

Request (example)

Request with cache and all content parameters (JSON in response is beautified).

GET /?url=http://example.com/&key=97e67gjhfb3f6a8dc3efcc853830d813134&content=tmh&cache=0 HTTP/1.1
Host: api.textracto.com
Response(example)
{ 
   "text”:”Example Text.”,
   "metas":{ 
      "title":"Blog - Example”,
      "type":"website",
      "url":"http://example.com/blog/",
      "description":"Modify your meta description by editing it right here",
      "image":" ",
      "updated_time":" ",
      "locale":"en_US",
      "site_name”:”Example”
   },
   "images_headlines":{ 
      "headlines":{ 
         "h1":[ 
            “Example.”
         ],
         "h2":[ 
            "Posts",
            "Recent",
            “Example”,
            “Test-Page”,
            "Meta"
         ],
         "h3":[ 

         ],
         "h4":[ 

         ],
         "h5":[ 

         ]
      },
      "images":[ 
         "http://example.com/wp-content/uploads/2015/12/s.png",
         "http://example.com/wp-content/uploads/2015/12/00x220.png",
         "http://example.com/wp-content/uploads/2015/12/s0.png",
         "http://example.com/wp-content/themes/zerif-lite/images/ish.png",
         "http://example.com/wp-content/themes/zerif-lite/images/n.png",
         "http://example.com/wp-content/themes/zerif-lite/images/teue.png"
      ],
      "links":[ 
         "http://example2.com",
         "http://example3.com",
         "http://example4.com/index.php",
         "http://example5.com/wp-content/themes/zerif-lite/",
         "http://example6.com/text-page",
         "http://example7.com/wp-content"
      ]
   }
}
Request

Standart request, without optional parameters.

GET /?url=http://www.bbc.com/news/election-us-2016-37104688&key=97e67gjhfb3f6a8dc3efcc853830d813134 HTTP/1.1
Host: api.textracto.com
Response
HTTP/1.1 200 OK 
Date: Wed, 17 Aug 2016 14:27:59 GMT
Server: WEBrick/1.3.1 (Ruby/2.3.1/2016-04-26)
Content-Type: application/json
Content-Length: 6464
X-Content-Type-Options: nosniff
Connection: close

{  
   "text":"

Republican presidential candidate Donald Trump has overhauled his campaign team for the second time in two months, with two new leaders.

Pollster Kellyanne Conway becomes campaign manager and Stephen Bannon of Breitbart News the CEO. Paul Manafort remains campaign chairman, but analysts say he has effectively been demoted.

Mr Trump told AP the new leaders were \"terrific people... they're champs\".

Mr Trump has seen his poll ratings slip since the party conventions last month.

He trails Democratic Party candidate Hillary Clinton both nationally and in key battleground states.

<h2>'I don't want to change'</h2>


The latest shake-up comes just 82 days before the election.

: \"I have known Steve and Kellyanne both for many years. They are extremely capable, highly qualified people who love to win and know how to win.\"

The Associated Press news agency said the details of the new hierarchy were hammered out at a lengthy senior staff meeting at Trump Tower on Tuesday and that more senior appointments were expected in the coming days.

At a campaign rally in Ohio on Tuesday, Mrs Clinton said the changes would make little difference to the race.

\"He can hire and fire anyone he wants... They can make him read new words from a teleprompter. But he is still the same man,\" Mrs Clinton said.

<h2>Analysis: Aleem Maqbool, BBC North America correspondent </h2>


Battered in the polls, Donald Trump has shaken up his campaign yet again, but don't expect a kinder, gentler candidate anytime soon. 

Many senior Republicans want him to rein in his impulsive outbursts and soften his rough edges to appeal to more voters, especially women and independents.  

However, Mr Trump's appointment of Stephen Bannon, an unabashed right-wing firebrand, to lead his campaign seems to signal that more controversy is ahead, not less.  

The reshuffle suggests that while he recognises a change is needed, the New York billionaire won't be the one changing.  

\"I don't wanna change. Everybody talks about, 'Oh well, you're gonna pivot, you're gonna' - I don't wanna pivot. I mean, you have to be you. If you start pivoting, you're not being honest with people,\" Mr Trump said.

The new appointments come ahead of the Trump team's first major TV advertisements, due to start this week.

Although Mr Manafort stays in his job, analysts say the new appointments, which come two months after , represent a demotion.

The Washington Post cited Trump campaign aides as saying Mr Trump respected Mr Manafort but felt \"boxed in\" by people \"who barely knew him\".

Mr Manafort, a former adviser to George HW Bush and Bob Dole, only joined the Trump campaign in March.

<h2>Stephen Bannon</h2>


The executive chairman of has himself described his role there as \"virulently anti-establishment\", views that have led the website into staunch support of Mr Trump.

 described Mr Bannon as \"the most dangerous political operative in America\" - a phrase Mr Trump's own website was happy to repeat in announcing his appointment.

He will temporarily step down from Breitbart to work on the campaign full time and is expected to lead a highly aggressive strategy.

Mr Trump has been pressed by some Republicans to tone down his fiery rhetoric in the wake of a number of controversial comments in the past two weeks and the subsequent drop in poll ratings.

But Mr Trump appears to want to stand by the campaign style that won him the Republican nomination.

He said on Tuesday: \"You know, I am who I am. It's me. I don't want to change.

\"Everyone talks about, 'Oh, well you're going to pivot, you're going to.' I don't want to pivot. I mean, you have to be you. If you start pivoting, you're not being honest with people.\"

<h2>Kellyanne Conway</h2>


Ms Conway has experience with political campaigns, having previously worked with former Vice-President Dan Quayle and former House Speaker Newt Gingrich.

In a Republican press release last month, Ms Conway was said to be widely regarded as \"an expert on female consumers and female voters\".

Her ability to accurately predict election results in the past has won her awards.

Mr Manafort has had a troubled week, following a report in the New York Times that ledgers in Ukraine showed he was earmarked for $12.7m (£9.8m) in undisclosed cash payments from the former pro-Russian government between 2007 and 2012. .

The Trump team's alleged pro-Russia links have been a key issue of the campaign, and the latest allegations sparked a call from the Clinton campaign for a full disclosure.

Opinion polls since the national conventions have made grim reading for the Trump team, both nationally and in key states.

The national lead for the Democratic candidate is currently between seven and eight points, the polls suggest.

 that no modern candidate trailing by this much three weeks after the conventions had won the election.

An opinion poll in the state of Virginia, , gave Mrs Clinton a 14-point lead there.

",
   "metas":{  
      "utf-8":null,
      "X-UA-Compatible":"IE=edge,chrome=1",
      "description":"Republican presidential candidate Donald Trump overhauls his election campaign team for the second time in two months, amid flagging polls.",
      "x-country":"de",
      "x-audience":"International",
      "CPS_AUDIENCE":"International",
      "CPS_CHANGEQUEUEID":"284407551",
      "og:title":"US election 2016: Trump overhauls campaign team again - BBC News",
      "og:type":"article",
      "og:description":"Republican presidential candidate Donald Trump overhauls his election campaign team for the second time in two months, amid flagging polls.",
      "og:site_name":"BBC News",
      "og:locale":"en_GB",
      "article:author":"BBC News",
      "article:section":"US Election 2016",
      "og:url":"http://www.bbc.com/news/election-us-2016-37104688",
      "og:image":"http://ichef-1.bbci.co.uk/news/1024/cpsprodpb/14407/production/_90815928_mediaitem90815927.jpg",
      "twitter:card":"summary_large_image",
      "twitter:site":"@BBCWorld",
      "twitter:title":"US election 2016: Trump overhauls campaign team again - BBC News",
      "twitter:description":"Republican presidential candidate Donald Trump overhauls his election campaign team for the second time in two months, amid flagging polls.",
      "twitter:creator":"@BBCWorld",
      "twitter:image:src":"http://ichef.bbci.co.uk/news/560/cpsprodpb/14407/production/_90815928_mediaitem90815927.jpg",
      "twitter:image:alt":"Donald Trump in Wisconsin, 16 Aug",
      "twitter:domain":"www.bbc.com",
      "apple-mobile-web-app-title":"BBC News",
      "application-name":"BBC News",
      "msapplication-TileImage":"http://static.bbci.co.uk/news/1.144.0556/windows-eight-icon-144x144.png",
      "msapplication-TileColor":"#bb1919",
      "cleartype":"on",
      "mobile-web-app-capable":"yes",
      "robots":"NOODP,NOYDIR",
      "theme-color":"#bb1919",
      "viewport":"width=device-width, initial-scale=1, user-scalable=1",
      "fb:admins":"100004154058350"
   },
   "images_headlines":{  
      "headlines":{  
         "h1":[  
            "US election 2016: Trump overhauls campaign team again"
         ],
         "h2":[  
            "Accessibility links",
            "BBC navigation",
            "News navigation",
            "'I don't want to change'",
            "Analysis: Aleem Maqbool, BBC North America correspondent ",
            "Stephen Bannon",
            "Kellyanne Conway",
            "\n        Share this story        About sharing\n    ",
            "Road to the White House",
            "Top Stories",
            "\n        \n        Features & Analysis\n    ",
            "News navigation",
            "BBC News Services",
            "Explore the BBC"
         ],
         "h3":[  
            "Tool of mystery",
            "Four-legged colleague",
            "Scent of death ",
            "Forgotten disaster",
            "Athletics capital",
            "7 days quiz",
            "From Aleppo to Ambleside",
            "The godfather",
            "Sprinting squabble"
         ],
         "h4":[  

         ],
         "h5":[  

         ]
      },
      "images":[  
         "http://static.bbci.co.uk/frameworks/barlesque/3.20.4/orb/4/img/bbc-blocks-dark.png",
         "http://ichef.bbci.co.uk/news/320/cpsprodpb/0F6F/production/_90815930_034759043-1.jpg",
         "http://stats.bbc.co.uk/o.gif?~RS~s~RS~News~RS~t~RS~HighWeb_Story~RS~i~RS~37104688~RS~p~RS~102423~RS~a~RS~International~RS~u~RS~/news/election-us-2016-37104688~RS~r~RS~0~RS~q~RS~0~RS~z~RS~8026~RS~"
      ],
      "links":[  
         "#page",
         "/accessibility/",
         "/id/status?ptrt=http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688",
         "http://www.bbc.co.uk/news/",
         "http://www.bbc.com/news/",
         "/sport/",
         "/weather/",
         "http://shop.bbc.com/",
         "http://www.bbc.com/earth/",
         "http://www.bbc.com/travel/",
         "http://www.bbc.com/capital/",
         "/iplayer/",
         "http://www.bbc.com/culture/",
         "http://www.bbc.com/autos/",
         "http://www.bbc.com/future/",
         "/tv/",
         "/radio/",
         "/cbbc",
         "/cbeebies",
         "/food/",
         "/iwonder",
         "/education",
         "/travel/",
         "/music/",
         "/arts/",
         "/makeitdigital",
         "/taster",
         "/nature/",
         "/local/",
         "#orb-footer",
         "http://search.bbc.co.uk/search",
         "/news",
         "#core-navigation",
         "/news/video_and_audio/international",
         "/news/world",
         "/news/uk",
         "/news/business",
         "/news/technology",
         "/news/science_and_environment",
         "/news/magazine",
         "/news/entertainment_and_arts",
         "/news/health",
         "/news/world_radio_and_tv",
         "/news/in_pictures",
         "/news/also_in_the_news",
         "/news/special_reports",
         "/news/explainers",
         "/news/the_reporters",
         "/news/have_your_say",
         "/news/world/us_and_canada",
         "/news/election/us2016",
         "mailto:?subject=Shared%20from%20BBC%20News&body=http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688",
         "http://www.facebook.com/dialog/feed?app_id=58567469885&redirect_uri=http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688&link=http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688%3FSThisFB",
         "https://twitter.com/intent/tweet?text=BBC%20News%20-%20US%20election%202016%3A%20Trump%20overhauls%20campaign%20team%20again&url=http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688",
         "https://uk.pinterest.com/pin/create/bookmarklet/?url=http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688&description=Republican%20presidential%20candidate%20Donald%20Trump%20overhauls%20his%20election%20campaign%20team%20for%20the%20second%20time%20in%20two%20months%2C%20amid%20flagging%20polls.&title=US%20election%202016%3A%20Trump%20overhauls%20campaign%20team%20again&media=http%3A%2F%2Fc.files.bbci.co.uk%2F14407%2Fproduction%2F_90815928_mediaitem90815927.jpg",
         "whatsapp://send?text=BBC%20News%20%7C%20US%20election%202016%3A%20Trump%20overhauls%20campaign%20team%20again%20-%20http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688%3Focid%3Dwsnews.chat-apps.in-app-msg.whatsapp.trial.link1_.auin",
         "https://www.linkedin.com/shareArticle?mini=true&url=http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688&title=US%20election%202016%3A%20Trump%20overhauls%20campaign%20team%20again&summary=Republican%20presidential%20candidate%20Donald%20Trump%20overhauls%20his%20election%20campaign%20team%20for%20the%20second%20time%20in%20two%20months%2C%20amid%20flagging%20polls.&source=BBC",
         "http://www.bbc.com/news/election-us-2016-37104688",
         "http://www.bbc.co.uk/faqs/questions/bbc_online/sharing",
         "/news/election-us-2016-37108732",
         "https://www.donaldjtrump.com/press-releases/donald-j.-trump-announces-major-campaign-hires",
         "http://www.bbc.co.uk/news/world-us-canada-36579467",
         "http://www.breitbart.com/",
         "http://www.bloomberg.com/politics/graphics/2015-steve-bannon/",
         "http://www.bbc.co.uk/news/election-us-2016-37080909",
         "http://www.nytimes.com/2016/08/16/upshot/the-state-of-the-clinton-trump-race-is-it-over.html",
         "https://www.washingtonpost.com/local/virginia-politics/trump-unpopularity-fuels-wide-lead-for-clinton-in-new-virginia-poll/2016/08/15/ea0e1540-6307-11e6-be4e-23fc4d4d12b4_story.html",
         "http://www.bbc.co.uk/help/web/sharing.shtml",
         "https://uk.pinterest.com/pin/create/bookmarklet/?url=http%3A%2F%2Fwww.bbc.com%2Fnews%2Felection-us-2016-37104688&description=Republican%20presidential%20candidate%20Donald%20Trump%20overhauls%20his%20election%20campaign%20team%20for%20the%20second%20time%20in%20two%20months%2C%20amid%20flagging%20polls.&title=US%20election%202016%3A%20Trump%20overhauls%20campaign%20team%20again&media=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fspecial%2F2015%2Fnewsspec_10857%2Fbbc_news_logo.png%3Fcb%3D1",
         "/news/world-us-canada-35457209",
         "/sport/olympics/36690965",
         "/news/world-latin-america-37127499",
         "/newsbeat/articles/37085042",
         "/news/world-europe-37109169",
         "/news/world-asia-india-37092728",
         "/news/world-australia-37091400",
         "http://www.bbc.com/news/world-us-canada-37121404",
         "/news/world-us-canada-37102762",
         "/news/magazine-37120920",
         "/news/world-middle-east-37049314",
         "/news/world-africa-37120388",
         "/news/blogs-trending-37107541",
         "/news/world/africa",
         "/news/world/asia",
         "/news/world/australia",
         "/news/world/europe",
         "/news/world/latin_america",
         "/news/world/middle_east",
         "/news/england",
         "/news/northern_ireland",
         "/news/scotland",
         "/news/wales",
         "/news/politics",
         "http://www.bbc.co.uk/news/business/market_data",
         "/news/business/markets",
         "/news/business/economy",
         "/news/business/companies",
         "/news/business-22434141",
         "/news/business-11428889",
         "/news/business/business_of_sport",
         "/news/business-12686570",
         "http://www.bbc.co.uk/news/10628994",
         "http://www.bbc.co.uk/news/help-17655000",
         "http://www.bbc.co.uk/news/10628323",
         "http://www.bbc.co.uk/news/20039682",
         "/terms/",
         "/aboutthebbc/",
         "/privacy/",
         "/privacy/cookies/about",
         "/guidance/",
         "/contact/",
         "http://advertising.bbcworldwide.com/",
         "/privacy/cookies/international/",
         "/help/web/links/"
      ]
   }
}

Need help? Feel free to contact us!