04 Oct 2016
I am seeing more examples of analytics at the API client and SDK level, providing more access to what is going on at this layer of the API stack. I'm seeing API providers build them into the analytics they provider for API consumers, and more analytic services from providers for the web, mobile, and device endpoints. Many companies are selling these features in the name of awareness, but in most cases, I'm guessing it is about adding another point of data generation which can then be monetized (IoT is a gold rush!).
As I do, I wanted to step back from this movement and look at it from many different dimensions, broken down into two distinct buckets:
- More information - More data than can be analyzed
- More awareness - We will have visibility across integrations.
- Real-time insights - Data can be gathered on real time basis.
- More revenue - There will be more revenue opportunities here.
- More personalization - We can personalize the experience for each client.
- Fault Tolerance - There are opportunities for building in API fault tolerance.
- More information - If it isn't used it can become a liability.
- More latency - This layer slows down the primary objective.
- More code complexity - Introduces added complexity for devs.
- More security consideration - We just created a new exploit opportunity.
- More privacy concerns - There are new privacy concerns facing end-users.
- More regulatory concerns - In some industries, it will be under scrutiny.
I can understand why we want to increase the analysis and awareness at this level of the API stack. I'm a big fan of building in resiliency in our clients & SDKs, but I think we have to weigh the positive and negatives before jumping in. Sometimes I think we are too willing to introduce unnecessary code, data gathering, and potentially opening up security and privacy holes chasing new ways we can make money.
I'm guessing it will come down to each SDK, and the API resources that are being put to work. I'll be aggregating the different approaches I am seeing as part of my API SDK research and try to provide a more coherent glimpse at what providers are up to. By doing this, I'm hoping I can better understand some of the motivations behind this increased level of analytics being injected at the client and SDK level.
See The Full Blog Post
10 Jun 2015
When I started API Evangelist, I knew I didn't want to use WordPress or other common CMS, so I developed my own API, and page and blogging CMS. During the latest migration of my internal API infrastructure, I'm rebuilding everything as a single stack of APIs that I can use to operate API Evangelist. Part of this process is breaking down legacy systems, into the small possible unit of value, something I consider deeply as I rebuild each API.
It started with my link system, which I broke up into link, curation, keyword, and linkrot APIs. Now I'm eyeballing my legacy blog API, which historically I also use to publish news pieces to API.Report, and sections that I syndicate to white papers I have published. My goal with my blogging API is to get it down to its simplest possible functionality--publishing posts to my blogs.
With this in mind I forked my newer blog API, and labeled it my news API. Right now, my news API behaves very similar to my blog, but it already has new features like automated pulling of press releases, extraction of proper nouns, and a few other bells and whistles that make my news management workflow as painless as possible. Over time I may share some of the features with the blogging API, I think my news and analysis worlds will remain separate beasts, from here forward.
See The Full Blog Post
11 Mar 2015
I came across an interesting piece of technology today while doing new curation for API.Report. RASON, an interesting approach to API driven analytics and potential UI and visualization, that kind of resembles what I have been envisioning for one possible future. The analytics tool is created by a company called Frontline Systems, and I’ll let them articulate what it is:
RASON™ software is designed to make it much easier to create, test and deploy analytic models that use optimization, simulation, and data mining. RASON stands for Restful Analytic Solver™ Object Notation.
RASON targets analytical professionals, excel power users, and web app developers, but here is where it gets over my head, "Problems you can solve with the RASON service include linear programming and mixed-integer programming problems, quadratic programming and second-order cone problems, nonlinear and global optimization problems, problems requiring genetic algorithm and tabu search methods -- from small to very large." — sounds impressive to me!
I signed up and played with RASON a little bit, but it wasn't as intuitive as I hoped. I think I have a little more to learn about RASON. The RASON models are very cool, I like the airport hub, I just don’t have enough knowledge to make it work right now, however I’m digging the idea, and it reflects what I’ve been seeing in my head when it comes to defining API driven analysis--when you connecting that with API generated visualizations, hypermedia, spreadsheets, APIs.son, and more—I start to get a little too excited.
Anyhoo. Just sharing a new technology that I found. Once I learn more about RASON, hopefully I will be able to see where RASON fits into the bigger API life-cycle, and share a little more.
See The Full Blog Post
10 Jul 2014
I looked through 77 of the developer areas for federal agencies, resulting in reviewing approximately 190 APIs. While the presentation of 95% of the federal government developer portals are crap, it makes me happy that about 120 of the 190 APIs (over 60%) are actually consumable web APIs, that didn't make me hold my nose and run out of the API area.
Of the 190, only 13 actually made me happy for one reason or another:
Don't get me wrong, there are other nice implementations in there. I like the simplicity and consistency in APIs coming out of GSA, SBA, but overall federal APIs reflect what I see a lot in the private sector, some developer making a decent API, but their follow-through and launch severeley lacks what it takes to make the API successful. People wonder why nobody uses their APIs? hmmmmm....
A little minimalist simplicity in a developer portal, simple explanation of what an API does, interactive documentation w/ Swagger, code libraries, terms of service (TOS), wouild go a looooooooooooong way in making sure these government resources were found, and put to use.
Ok, so where the hell do I start? Let's look through theses 123 APIs and see where the real low hanging fruit for demonstrating the potential of APIs.json, when it comes to API discovery in the federal government.
Let's start again with the White House (http://www.whitehouse.gov/developers):
Only one API made it out of the USDA:
Department of Commerce (http://www.commerce.gov/developer):
- Census Bureau API - http://www.census.gov/developers/ - Yes, a real developer area with supporting building blocks. (Update, News,( App Gallery, Forum, Mailing List). Really could use interactive document though. There are urls, but not active calls. Would be way easier if you could play with data, before committing. (B)
- Severe Weather Data Inventory - http://www.ncdc.noaa.gov/swdiws/ - Fairly basic interface, wouldn’t take much to turn into modern web API. Right now its just a text file, with a spec style documentation explaining what to do. Looks high value. (B)
- National Climatic Data Center Climate Data Online Web Services - http://www.ncdc.noaa.gov/cdo-web/webservices/v2 - Oh yeah, now we are talking. That is an API. No interactive docs, but nice clean ones, and would be some work, but could be done. (A)
- Environmental Research Division's Data Access Program - http://coastwatch.pfeg.noaa.gov/erddap/rest.html - Looks like a decent web API. Wouldn’t be too much to generate a machine readable definition and make into a better API area. (B)
- Space Physics Interactive Data Resource Web Services - http://spidr.ngdc.noaa.gov/spidr/docs/SPIDR.REST.WSGuide.en.pdf - Well its a PDF, but looks like a decent web API. It would be some work but could turn into a decide API with Swagger specs. (B)
- Center for Operational Oceanographic Products and Services - http://tidesandcurrents.noaa.gov/api/ - Fairly straightforward API, Simple. Wouldn’t be hard to generate interactive docs for it. Spec needed. (B)
Department of Education:
- Department of Education - http://www.ed.gov/developers - Lots of high value datasets. Says API, but is JSON file. Wouldn’t be hard to generate APIs for it all and make machine readable definitions. (B)
- Energy Information Administration - http://www.eia.gov/developer/ - Nice web API, simple clean presentation. Needs interactive docs. (B)
- National Renewable Energy Laboratory - http://developer.nrel.gov/ - Close to a modern Developer area with web APIs. Uses standardized access (umbrella). Some of them have Swagger specs, the rest would be easy to create. (A)
- Office of Scientific and Technical Information - http://www.osti.gov/XMLServices - Interfaces are pretty well designed, and Swagger specs would be straightforward. But docs are all PDF currently. (B)
Department of Health and Human Services (http://www.hhs.gov/developer):
Food and Drug Administration (http://open.fda.gov):
Department of Homeland Security (http://www.dhs.gov/developer):
Two losse cannons:
Department of Interior (http://www.doi.gov/developer):
- Bureau of Land Management - Geocommunicator - http://www.blm.gov/nils/GeoComm/home_services.html - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- Bureau of Land Management - GGeocommunicator Map and Web Services - http://www.geocommunicator.gov/GeoComm/services.htm - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- Landscape ARCGIS Server - http://www.landscape.blm.gov/ArcGIS/rest/services - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- Bureau of Ocean Energy Management - BOEM ARCGIS Server - http://gis.boemre.gov/arcgis/sdk/rest/ - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- Fish and Wildlife Service - Environmental Conservation Online System web services - http://ecos.fws.gov/tat_services/ - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- Fish & Wildlife Service ARCGIS Service - http://gis.fws.gov/arcgis/rest/services - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- National Park Service - ArcGIS Server REST API - http://mapservices.nps.gov/arcgis/sdk/rest/index.html?query.html - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- US Geological Survey - Eastern Geographic Science Center Map Web Services - http://sscweb.gsfc.nasa.gov/WebServices/ - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- Map-A-Planet: Web Map Service - http://www.mapaplanet.gov/explorer/help/wmsUserDoc.html - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- National Atlas Web Map Services - http://nationalatlas.gov/infodocs/webservices.html - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- NationalMap.gov Web Services - http://services.nationalmap.gov/ - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- ScienceBase API - https://my.usgs.gov/confluence/display/sciencebase/ScienceBase+Item+Services - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- Streamstats web services - http://streamstatsags.cr.usgs.gov/webservices/wsui.htm - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
- Water Quality Portal Web Services - http://www.waterqualitydata.us/webservices_documentation.jsp - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
Department of Justice (http://www.justice.gov/developer):
- Department of Labor - http://developer.dol.gov/ - I love their developer area. They have a great API, easy to generate API definitions. (A)
- Bureau of Labor Statistics - http://www.bls.gov/developers/ - Web APIs in there. Complex, and lots of work, but can be done. API Definitions Needed. (B)
Department of State (http://www.state.gov/developer):
Department of Transportation (http://www.dot.gov/developer):
Department of the Treasury (http://www.treasury.gov/developer):
Veterans Affairs (http://www.va.gov/developer):
Consumer Finance Protectection Bureau:
Federal Communications Commission (http://www.fcc.gov/developers):
- Federal Reserve Bank of St. Louis - http://api.stlouisfed.org/ - Good API and area, would be easy to generate API definitions. (B)
General Services Administration (http://www.gsa.gov/developers/):
- American Job Center Resource API - http://jobcenter.usa.gov/apis - Good API and area, would be easy to generate API definitions. (B)
- BusinessUSA Resource Access API - http://business.usa.gov/apis - Good API and area, would be easy to generate API definitions. (B)
- Citizen Topics API - http://www.usa.gov/About/developer-resources/social-media-registry.shtml#tags - Good API and area, would be easy to generate API definitions. (B)
- Data Center Consolidation API - https://explore.data.gov/developers/docs/federal-data-center-consolidation-initiative-fdcci-data-center-closings-2010-2013 - Data.gov, simple API, Easy to create API definition. (B)
- Federal Agency Directory API Documentation - http://www.usa.gov/About/developer-resources/federal-agency-directory/index.shtml - Data.gov, simple API, Easy to create API definition. (B)
- Government Jobs API - http://search.digitalgov.gov/developer/jobs.html - Good API and area, would be easy to generate API definitions. (B)
- Domains API - https://explore.data.gov/developers/docs/federal-executive-agency-internet-domains - Data.gov, simple API, Easy to create API definition. (B)
- Go.USA.gov API - https://go.usa.gov/api - Good API and area, would be easy to generate API definitions. (B)
- Mobile App Gallery API Documentation - http://www.usa.gov/About/developer-resources/mobile-app-gallery/index.shtml - Good API and area, would be easy to generate API definitions. (B)
- MyUSA Citizen API - https://my.usa.gov/developer/ - Interesting API. Not Data. User-based. Need to look at oAuth! This needs more attention. (B)
- MyGov Discovery API - http://discovery.my.usa.gov/ - OK API and area, would be easy to generate API definitions. (B)
- Per Diem API - http://www.gsa.gov/portal/content/162379 - Good API and area, would be easy to generate API definitions. (B)
- Product Recall Data API - http://search.digitalgov.gov/developer/recalls.html - Good API and area, would be easy to generate API definitions. (B)
- Social Media Registry API - http://www.usa.gov/About/developer-resources/social-media-registry.shtml - Good API and area, would be easy to generate API definitions. (B)
National Aeronautics and Space Administration http://open.nasa.gov/developer:
Couple more loose cannons:
Recovery Accountability and Transparency Board (http://www.recovery.gov/arra/FAQ/Developer/Pages/default.aspx):
Small Business Administration (http://www.sba.gov/about-sba/sba_performance/sba_data_store/web_service_api):
Last but not least.
That is a lot of potentially valuable API resource to consume. From my perspective, I think that what has come out of GSA, SBA, and White House Petition API, represent probably the simplest, most consistent, and high value targets for me. Next maybe the wealth of APis out of Interior and FDA. AFter that I'll cherry pick from the list, and see which are easiest.
I'm lookig to create a Swagger definition for each these APIs, and publish as a Github repository, allowing people to play with the API. If I have to, I'll create a proxy for each one, because CORS is not common across the federal government. I'm hoping to not spend to much time on proxies, because once I get in there I always want to improve the interface, and evolve a facade for each API, and I don't have that much time on my hands.
See The Full Blog Post
04 Mar 2014
I’m working with the Cashtie API to pull together an API evangelism strategy for the payments API. As I pull together, it seems like a great opportunity to share with you. Who knows, maybe you can use some of the elements in your own API strategy.
Next up is what I call "landscape analysis", which is about monitoring the blogs, and other social network activity of the sector(s) you are evangelizing your API within:
- Competition Monitoring - Evaluation of regular the activity of competitors, visiting their sites, and getting to know their customers.
- Industry Monitoring - Evaluation of relevant topics and trends of overall industry, industries that you are evangelizing within.
- Keywords - Established a list of keywords to use when searching for topics at search engines, QA, forums, social bookmarking and social networks—this should dovetail nicely into any SEO work your doing. You are doing SEO work?
- City and Country Targeting - Target specific markets with your evangelism. Get to know where your existing developers and target developers reside, and get local--be on the ground.
- Social Media Monitoring - Have a solid strategy and platform for pulling relevant data from Twitter accounts, #hashtag conversations, Github profiles, and other relevant platforms to use in your landscape analysis.
The "landscape", is the space you are looking at each and every day out your virtual window, as well as what your existing and target developers will see each day. It is your job as an evangelist or advocate to know this space well, understand all the elements of the landscape, key players, relationships, and be a contributing member to this landscape.
Lanscape analysis should be something you do every day, weaving it into your regular social media, and other online activities. Remember you aren’t targeting this landscape of companies and users, you are looking to participate in the conversation, while also bring your API resources to the table.
See The Full Blog Post
31 Jan 2014
The public recently found out that the NSA uses leaky applications such as Angry Birds to track on and find out more about every day citizens around the world. There is a wealth of data available regarding the potential of Angry Bird users, where they travel to, how they spend their leisure time, and potentially their understanding of physics.
One very important application for this type of surveillance of citizens is in the area of crowd control, and protest analysis. By tracking users ability to successfully target birds, and fling pings in Angry Birds, you can quickly assess which users will potentially be the most successful in throwing of rocks and other objects during a protest.
When you couple this profile data, with the ability to target these individuals using their cell phone signal in a crowd, you can quickly identify the individuals in a crowd who will potential cause the most damage , as well as injury to law enforcement if things get ugly. However in this scenario, rather than throwing of virtual pigs at birds, protesters will be hurling other objects at angry pigs.
Using big data analysis, law enforcement can easily target these threatening individuals in a crowd, conduct an initial sweep of the crowd and eliminate the threat to law enforcement. This will ensure that large crowds and protests, when they turn violent will minimize the threat to officers and potentially reduce property destruction.
See The Full Blog Post
23 Jul 2011
provides an RESTful API for text analysis, using oath for authentication.
Using the Saplo API
you can perform text analysis such as entity tagging, related articles, contextual recognition and sentiment anlysis.
Saplo provides a free account for non-commercial use with a limit of 240 calls per hour, and also a premium plan with pricing based upon the amount of text processed.
Saplo API for text analysis
is a perfect tool for the API Stack.
See The Full Blog Post
21 Mar 2011
, a natural language processing
service announced a new sentiment analysis
layer to its existing text-mining Software as a Service (SaaS).
The service can be used by online publishers, news aggregators, and contextualadvertising
firms to understand and monetize online content.
The cloud-based text mining platform provides document-level, entity-level, and keyword level sentiment ming, in addition to support for negation handling, sentiment amplifiers or diminishers, slang and typos.
These features make the text mining API usable for analyzing and understanding user-generated social media content as well as editorialized text.
Sentiment analysis is available immediately to all AlchemyAPI users, more information is available at the AlchemyAPI web site
See The Full Blog Post
03 Dec 2010
A project I'm moving off the back-burner and into the public domain
is a project I called Tech Blogger Analysis. My GF is Audrey Watters @ ReadWriteWeb, so I get exposed to an authors daily perspective in the tech blogging world.
I started harvesting all the RSS feeds from ReadWriteWeb
, and GigaOm
Upon harvest I would organize the post and curate author details, tags, etc. I had a whole bunch of scripts I'd run to clean up, organize and centralize meta data.
I would then pull the tweets for each post daily using BackType.
I would then build profiles for each twitter user with their # of followers, tweets etc....as well as their Trustrnk using InfoChimps.
When I started I was profile the individual blogs, and quickly saw the true value is in the tech blogger and their brand. I think there is an important story here:
- What topics to they cover most? Specialize in?
- Who has the strong brand? Number of Followers? Importance in certain topics.
- Who are their influential followers? Loyalty? Trustrnk?
- What do their followers pay attention to?
There is a lot of information here. I had the data harvesting for last 6 months, it was being refined and had a crude web interface setup. I wanted to provide a directory of Tech Bloggers and their brands with real-time stats on what they are writing on their personal and professional blogs. Think Postrank, BackType, Crunchbase for Bloggers.
I think the future of tech journalism is about the individual writers and their brands. I think writers would find this information valuable. I also think industry leaders would find this information valuable also. I see a lot of corporations investing in tech writers and their brand. I see writer brands becoming just as, or more important than the domain based brands we see today.
I would like to restart this project, but just can't afford the EC2, RDS and bandwidth charges if I'm not moving forward. If you are interested, you can contact me.
See The Full Blog Post
11 Feb 2008
Amazon just launched a new analysis tool for Amazon S3 called, S3Stat. This tool uses the log files generated by S3, analyzes them using Webalizer
, and generates a variety of reports. Take a look at the sample reports
to learn more. There's a one-month free trial and usage after that costs just $2 per month. Take a look at the pricing plan
to learn more. While you are on the site you may want to take a look at their list of S3 resources
See The Full Blog Post
24 Jan 2008
After posting about cellular and mobile barcodes yesterday I received a comment from a guy who started an online magazine on the subject. 2d-code @ http://2d-code.co.uk/
From their home page about the magazine:
In Japan QR codes are found on everything from business cards to fresh lettuce. Now they are coming to the West and advertising and promotion will never be the same again. 2d Code will keep you informed as it happens, with news, views and analysis.
If you are interested in the QR code phenomena you can participate with us in several ways.
- Mention or write about this website on your blog
- Send us your sightings of QR or 2d barcodes
- At the bottom of each individual news item there is an opportunity for you to comment or leave a message
- At the top of each individual news item you can email the item to a friend or colleague.
Great resource to introduce and follow the subject of mobile two dimensional bar codes. I highly recommend you check it out and subscribe to their RSS Feed.
I really think this is an under-realized technology and topic in this country, and if we learn from the Japanese on how they use we can solve that problem off our online / offline lives and how that relates to social media and online marketing campaigns
See The Full Blog Post