What’s Monitio
Monitio is an AI-powered real time multilingual media intelligence platform developed and owned by Priberam. For that purpose, Monitio’s bot ingests content from thousands of web feeds across the World.
How Does it work
Monitio’s bot primarily uses RSS and sitemaps (currently more than 97% of the Web sources). Because the feeds are mainly from news outlets, the bot usually doesn’t ingest historic information. For instance, typically for a full news sitemap that has no timestamps, the bot first gets all the urls recursively without scraping them and only on subsequent passes it starts scraping the new urls to get the content.
Requests are always made from the same range of IPs. Monitio’s bot doesn’t use rotating/distributed proxies to bypass IP blocking.
Monitio’s bot complies with robots.txt directives and dynamically adjusts the frequency at which it revisits a feed. For big media outlets that publish new content throughout the day it can fetch the RSS or sitemap every 2 or 3 minutes but for a blog that posts only four or five times a day, it will do it perhaps every 2 or 3 hours.
For the minority of web sites that don’t provide a RSS feed or sitemap, Monitio’s bot crawls one or more pages to get the list of new articles published before proceeding to the content scraping.
User Agent
PriberamBot/1.0 Media Monitoring platform – MONITIO
IP
148.69.128.1
ReverseDNS
bot.monitio.com
Licensed news content
Besides the technical and policy compliance, Monitio is also copyright compliant. Priberam as signed content licensing agreements with collective copyright management organizations like CEDRO in Spain, NLA Media Access in the United Kingdom and Visapress in Portugal to use the content from media web sites in these countries. Priberam is extending these content licensing agreements to other countries directly or through partnerships. For instance, Monitio is ingesting content from broadcasters like Deutsche Welle, in Germany, press agencies like EFE, in Spain and LUSA in Portugal, at their request.
Social Media
Besides web news Monitio also signed agreements with platforms like Facebook and X (former Twitter) to ingest social media content from news outlets.
Other Content
For research purposes Monitio’s bot also ingests other news feeds around the world in multiple languages. The news outlets for which Priberam has still no licensing agreement can’t be read by the users in Monitio. In the case, Monitio behaves just like any search engine and only presents the title of the news, a snippet and a link to the original web site.
Affiliations
Priberam is a member of FIBEP, the World’s Media Intelligence Association with over 130 corporate members from more than 60 countries.
Priberam is also a founding member of the Centre for Responsible AI and is committed to build products that are fair and that includes being transparent on how we get the content used in our products.
