Contributing to the AMP Project

Editor’s note: The following was originally posted by Adam Silverstein, Lead Web Engineer at 10up on the Google Open Source Blog.

I started my web career building websites for small businesses on WordPress, so when I decided to begin contributing to open source, WordPress was a natural place to start.

Now I work at the digital agency 10up, where I am a part of our open source team. We build popular sites like FiveThirtyEight where having the best possible AMP experience is critical. However, bringing FiveThirtyEight’s AMP version up to parity with the site’s responsive mobile experience was challenging, in part because of advanced features that aren’t directly supported in AMP.

One of those unsupported features was MathML, a standard for displaying mathematical formulas on the web. To avoid a clumsy work around (amp-iframe) and improve our presentation of formulas, I proposed a native `amp-mathml` component which could display formulas inline. Contributing improvements “upstream” to open source projects – especially as we encounter friction in real-world projects – is a core value at 10up and important to the health of the web. I expected that I could leverage the same open source MathJax library we used on the responsive website for an AMP implementation. Contributing this component would strengthen my understanding of AMP’s internals while simultaneously improving a client site and enabling the open MathML standard on any AMP page. Win, win, win!

I started by opening an issue on the amphtml repository, describing MathML and proposing a native `amp-mathml` component. Justin Ridgewell from the AMP team immediately responded to the issue and asked Ali Ghassemi to track it. I offered to help write the code and received an enthusiastic response, encouraging me and assuring me that the team would be available on GitHub and in Slack to answer any questions.

This warm welcome gave me the confidence to dive in, but ramp up was daunting. The build tools and coding standards were quite different from other projects I work on and setup required some editor reconfiguring and reflex retraining. Getting the unit test to run on my system required tracking down and installing some missing dependencies.

Fortunately, AMP’s project documentation is thorough, and Ali guided me through the implementation, pointing me to existing, similar samples in the project. I already knew how to use JavaScript to render formulas with MathJax – my challenge was building an AMP component that ran this code and displayed it inline.

After a few days of concerted effort, I built a proof of concept and opened a pull request. The real fun began as I refined the approach and wrote documentation with help from the team. The team’s active engagement helped the process move along rapidly. Amazingly, the pull request was merged one week later, and today amp-mathml is live in the wild. FiveThirtyEight is already using the new, native implementation.

From opening the issue all the way to the merge of my pull request, I was impressed by the support and encouragement I received. Ali and honeybadgerdontcare provided regular reviews and thorough suggestions on the pull request when I pushed iterations. Their engagement throughout the process made me and my work feel valued, and helped me stay motivated to continue working on the feature.

Adding MathML to AMP reminded me why I find so much joy and professional growth in contributing to open source projects. I have a better understanding of AMP from the inside out, and I was welcomed into the project’s community with wide open arms. I’m proud of my contribution, and ready to tackle new challenges after seeing its success!

Posted By Adam Silverstein, Lead Web Engineer at 10up, and AMP Project contributor

Contributing to the AMP Project

The Shadow Reader, Improved

We made the Shadow Reader even faster – and friendlier to search engines!

We created the Shadow Reader to demonstrate how AMP pages can be used within a Progressive Web App (PWA) (read our announcement post for more context). The site repurposes existing articles from The Guardian into an immersive news reader experience. More than just a demo, it’s intended to be a fully functional site. It contains the end-to-end code needed to effectively combine AMP and PWA… it’s ready for production!

 

SEO for JS-generated content

As in any self-respecting single page application, the Shadow Reader’s initial HTML payload is small. It’s a thin app shell that loads quickly, giving the user something to look at while JavaScript loads the main content. This approach makes for a fine user experience!

Unfortunately, it can also present a challenge to search engines. Google will try to execute JavaScript in order to index what ultimately appears to the user, not just the initial HTML. But many search engines don’t or do so unreliably. In other words, it’s not safe to depend on a search engine to successfully execute your JavaScript. And if a search engine sees only the app shell, minus most or all of the content, it won’t be able to properly index the page.

Wouldn’t it be nice if the Shadow Reader’s article pages were served to search engines with text included right in the HTML?  And wouldn’t it be amazing if that process didn’t slow down rendering, but instead gave us a way to serve those pages to new users in less than a second?

Turns out we can do both of those by serving the AMP version of articles to new users!  After all, a web crawler appears to a server as a new user, too. So… how did we do it?

 

AMP⇒PWA

We did this by implementing an AMP⇒PWA pattern. Here’s how it works!

For a new user:

  • When a new user visits an article page, we serve the AMP version of the article.
  • The AMP uses <amp-install-serviceworker> to load and install the service worker.
  • The service worker loads and caches the app shell.
  • On the next page navigation, the service worker is in control – and so it gently brings the user into the PWA on that next page.

For an existing user, we simply serve the PWA.

That’s how our site can treat new users and existing users differently at the same URL. For existing users, the service worker is installed. And, when the service worker sees an article URL, it serves its cached version of the PWA.

How does this play out in the Shadow Reader?  Let’s say a user first visits this article page:

https://amp.cards/theguardian/us/amazing_article

Seeing an article URL, the server returns the AMP version of the article, but one that installs a service worker when the article is loaded. The service worker, using the Workbox library, contains this line:

workboxSW.router.registerNavigationRoute('index.html')

This means, whenever the user navigates to a new URL on this domain, the service worker sees that request, and instead of passing it along to the server, it simply serves its cached version of index.html. That’s our PWA.

So if the user next clicks on a link to

https://amp.cards/theguardian/us/another_article

the service worker serves the cached PWA HTML. But the URL is unchanged!  So when the PWA looks at the URL to parse out what article is being requested, it sees the link the user requested, and it can load the proper article into the PWA.

Thereafter, any time the user requests a Shadow Reader link, the service worker is already installed, and it serves the cached PWA.

Since a web crawler won’t allow us to install a service worker, a web crawler always gets served the AMP article.

 

Here’s that flow in a lovely diagram:

For a new user:

  1. The browser requests an article URL from the server. The server returns an AMP version of the article that includes <amp-install-serviceworker>.
  2. AMP’s serviceworker JS causes the browser to request the service worker. The server sends the service worker JS to the browser. The browser installs the service worker and starts it up.
  3. The service worker sends the server a request for the PWA app shell. The server sends those resources to the service worker, which caches them.

For an existing user:

  1. The browser sends a request for an article URL. This request is intercepted by the service worker. The service worker returns the cached PWA to the browser.
  2. The PWA requests the AMP article. This requests reaches the server, which returns the AMP article to the PWA. The PWA processes and displays that article.

Remember, a web crawler is always a new user!

 

What’s next?

Now that the Shadow Reader’s got its own server, we’ve got some new TODOs:

  • In the future, we could forgo YQL altogether, simply using the Guardian’s RSS feed directly.
  • We also should replace the Guardian’s top nav links with Shadow Reader links.
  • We ask the AMP Cache to download and run the entire Shadow Reader in an iframe: <amp-install-serviceworker data-iframe-src=”https://amp.cards/index.html“>. It might be kinder to the casual user to specify a smaller page instead.
  • Backend.js now gets used in the server as well as the front end, and the way we do that is a bit hacky. Perhaps we should refactor our code to use ECMAScript modules?

Please try this out, check out the code on github, and let us know what you think!  We’re curious about how you’re trying out AMP/PWA patterns on your own site, and we’d love to get your ideas for Shadow Reader improvements.

 

Posted by Ben Morss, Developer Advocate, Google

The Shadow Reader, Improved

New in AMP: Q2, 2018 Edition

Monetization support for AMP stories

AMP stories now supports ‘Publisher placed ads’, the ability for a publisher to serve ads for which they can control delivery sales and delivery. Ad Server support in DoubleClick for Publishers (DFP) is planned for the end of June. We urge other ad networks that are interested in adding monetization support to AMP stories to reach out to us.

6

See the details related to monetization related to AMP stories here or play with an example on AMPbyExample.com.

New user control tools in AMP

We have launched the new <amp-consent> component to help publishers implement user controls for customizable notice, choice, and consent flows. You can find all the usage and integration details related to the launch in this blog post.

Relatedly, a new component, <amp-geo>, makes it easy to vary small sections of web content for users based on an approximation of the users’ country-level location, similar to the level of an ISO Country Code. This can be used with <amp-consent> to change the behavior of <amp-consent> based on users’ approximate location. Check out an example to see how the two components integrate.

AMP Date Picker (Experimental)

The <amp-date-picker> component is a calendar-style interactive date picker for form input, and it’s officially available to use on sites as a document-level experiment. This means that developers can use the component as long as they also include this temporary tag: <meta name=”amp-experiments-opt-in” content=”amp-date-picker”>. Check out the sample on  AMP by Example to see what it can do, and determine if it’s right for your site.

Note: The reason for this intermediate stage of release (a document-level opt-in) is to collect feedback from developers and users and to identify any issues before we consider the component stable enough for a full production release. So expect some rough edges for now, let us know if you have any feedback, and stay tuned for the full release later this quarter.

2

Introducing the image lightbox gallery

Since AMP launched, developers have been asking for an easy way to link <amp-carousel> with <amp-image-lightbox> so users could experience an immersive view of content in carousels. While it’s been possible to combine numerous components (like <amp-bind>) to make something like this work, it hasn’t been easy.

The <amp-lightbox-gallery> component provides an immersive media experience that can stand on its own or can be integrated with the amp-carousel. The component can be applied to a standalone image or to a group of images so that when a user taps on these images, they enter an immersive view where they can seamlessly swipe from image to image. When <amp-lightbox-gallery> is applied to an <amp-carousel>, all images in the carousel inherit the lightbox gallery behavior so that swipes between items in the immersive view are synced with those in the lightbox, to create a smooth, intuitive gallery experience.

3

New effects for amp-fx-collection

We’re just starting to build out <amp-fx-collection>, a suite of easy-to-use visual effects that developers can use to make their sites more engaging. So far, we’ve released two effects: parallax, which lets you easily configure page elements to move at different speeds relative to the scrolling speed of the document; and fade-in, which applies a simple and configurable fade-in effect when the user scrolls content into view. Over the next few weeks, we’re planning to launch fly-in as well, which is currently available as an experiment. Check out the new features on AMP by Example and let us know what you think.

Ability to upload files

File upload is an important capability for many sites.  For instance, some e-commerce sites incorporate custom design uploads into their purchase flows. To support this functionality we enabled file upload support in forms through <input type=”file”>(essentially the same way developers would do this in a non-AMP page, with the additional requirement that the form uses a secure XHR submit). And now that AMP offers this basic support, we’re working on a more advanced API that will allow developers to better communicate to users the status of file uploads (bytes uploaded, % progress bars, etc).

The best of the rest

  • Read more about the progress on web packaging to help get rid of google.com/amp URLs when AMP documents are loaded from the Google AMP cache.
  • A number of talks and features were delivered at Google I/O 2018. You can read the summary of the AMP team’s presence at I/O here.

Upcoming features worth a shout

* * *

Thanks to the AMP development community for your work and feedback. As always, please let us know if you have any issues or feature requests.

Posted by Vamsee Jasti, Product Manager, AMP Project

New in AMP: Q2, 2018 Edition

AMP at Google I/O: Strengthening the AMP ecosystem

For the latest AMP project updates, subscribe to our new newsletter!

A strong network of AMP contributors and collaborators has been critical to achieving the project’s mission of building a user-first web. At Google’s I/O developer conference this week, members of the AMP team are sharing recent ways the AMP community has come together to improve key elements of the AMP ecosystem.

AMP has progressed a lot in the past year, especially thanks to the help and input of over 500 contributors. Components like amp-bind and amp-position-observer provide developers the tools to create their own rich interactivity on their AMP pages, rather than relying on more narrowly defined components. Along with these low-level frameworks, AMP has been focused on making it easier to create complex, immersive interactions. amp-fx-collection bundles some of the behaviors like parallax scrolling trivial to implement; amp-date-picker, now available as an experiment, offloads the complexity of a fully-featured date picker to a straightforward component; and with amp-lightbox-gallery, also available as an experiment, any developer can easily provide users with a seamless, immersive, media gallery.

In fact, AMP’s evolution has made it a viable solution to build entire websites. Major e-commerce companies like AliExpress have seen sizable business wins by launching a single all-AMP-driven mobile site. In particular AliExpress increased their conversion rate for non-search traffic by 31%. Combining AMP with PWA is also a powerful usage pattern, with sites like BMW.com building AMP pages within a PWA shell to load custom-built, all-AMP content for a seamless and fast user experience. BMW.com has seen 30% higher click through rates to national websites and 26% more mobile users overall. Finally, news content sites like Tencent who recently launched their news site built entirely with AMP saw a 2x increase in time on site and a 3.5x increase in page views per session.

WordPress
Support for AMP in the CMS ecosystem is critical to ensuring it’s easy to author content on the web. WordPress is one of the most popular tools for authoring web content, so Automattic, XWP, and Google are working together to advance the WordPress AMP plug-in. The recently launched 0.7 version includes native AMP support, allowing the creation of entire sites with AMP with the standard WordPress content creation workflow. This work includes the development of two full themes with native AMP support: News and Adventures (based on the AMP Start travel template). Ongoing work continues with a focus on adding AMP support to all core themes, and improving the integration of the AMP plugin in the development workflow in WordPress. The next major release will be v1.0 and is already in active development.

image2Native AMP ‘Adventures’ theme in WordPress

AMP URLs
We recognize the way platforms display AMP URLs are a concern for many, and earlier this year we shared our plans to improve AMP page URLs from Google Search. Since then, the AMP team has been working with Chrome and the web community to allow AMP URLs to point to a publisher’s domain when served from the cache. By utilizing the signed exchanges component of emerging Web Packaging technology, web content can be bundled and allow other parties to distribute it, while keeping the integrity guarantees of HTTPS. AMP project collaborators Food Network and Pinterest have built demos to showcase the capabilities of Web Packaging; one is shown below. To make this process easier, they used a new set of tools available at https://github.com/ampproject/amppackager. To learn more about the AMP team’s work on Web Packaging, see this complete blog post.

webpackagingA demo using Web Packaging with an AMP page from Google Search

AMP Stories
The possibilities of AMP extend beyond typical web pages, which is why we’ve also been collaborating with platforms and publishers on new, interactive formats like AMP stories. Content consumption on mobile is changing, with engaging, fullscreen storytelling formats becoming increasingly popular. While the web has all the capabilities to create content like this, the technical investment can be challenging. So to help meet this need, we are working with publishers to build the AMP stories format, a rich set of web components for storytelling on mobile. And while AMP stories are designed for mobile, they also work great on desktop. We’re continuing to prepare the format for a full release, but to learn more and try the developer preview now, visit ampproject.org/stories.

Examples of AMP stories from various publishers

AMP for Email
Additionally, we’re thinking about new ways to allow the broader web ecosystem to benefit from AMP. The AMP format is ideal for embedding content in other web-based experiences because it is performant, secure, and functional. While not immediately obvious, email is a great use case to enable actionable and interactive capabilities while maintaining the controls needed in an email client. To bring this experience to life, we’ve been collaborating with companies like Pinterest and Zillow. The AMP for email demos they have built showcase a new type of functionality that AMP facilitates inside the inbox.

image1Pinterest using AMP to enable interactivity inside an email

Most importantly, we want to thank the more than 500 contributors and additional collaborators we’ve worked with for helping to create a more user-friendly web. There are 6 billion AMP pages on the web, and they only exist through the work of the community that has come together to build the project. And we’re excited about the new capabilities and use-cases for AMP, which are brought to life by the folks working directly with the core AMP team. To learn about the latest in the AMP community, subscribe to our new newsletter, and to see all of the AMP talks at Google I/O 2018, check out the AMP YouTube channel.

Posted by Matt Ludwig, Marketing Lead for AMP at Google

AMP at Google I/O: Strengthening the AMP ecosystem

A first look at using web packaging to improve AMP URLs

One of the top pieces of feedback that people share about AMP is about the “google.com/amp…” URLs that are used when linking to a piece of AMP content in Google Search. A couple of months ago, the AMP team at Google outlined a plan to display better AMP URLs, and today we’d like to share progress on this effort.

Our approach uses one component of the emerging Web Packaging technologies—technologies that also support a range of other use cases. This component allows a publisher to sign an HTTP exchange (a request/response pair), which then allows a caching server to do the work of actually delivering that exchange to a browser. When the browser loads this “Signed Exchange”, it can show the original publisher’s web origin in the browser address bar because it can prove similar integrity and authenticity properties as a regular HTTPS connection.

Addressing this issue is great for users and publishers alike. Users see URLs that are consistent with their expectations based on the publisher identified on the search results page. Publishers benefit from retaining their brand in the address bar and the instant loading, thanks to pre-fetching.

To confirm the proposed tech works in practice, the AMP Project worked with two partners, Pinterest and Food Network, who signed their AMP content and published those signed exchanges to the web. To make this process easier, they used a new set of tools available at https://github.com/ampproject/amppackager.  Additionally a non-AMP–specific package tool is also available at https://github.com/WICG/webpackage/tree/master/go/signedexchange.

The Chrome team has built enough Signed Exchange support for developers to try it out. Starting with Chrome 67 on Android—in Beta channel at the time of writing—you can enable the experimental “Signed HTTP Exchange” flag under chrome://flags to use Web Packaging’s signed exchanges. In parallel with this experimental implementation, the Chrome team has also been collecting feedback from members of standards bodies, other browser vendors, security experts, and publishers and web developers to refine and improve the Web Packaging specifications.

Last, to tie everything together, the Google Search team has implemented a version of Google Search that illustrates the end-to-end flow. When a signed exchange is available, instead of linking to an AMP page served from Google’s AMP Cache, Google Search links to a signed AMP page served from Google’s cache.

webpackaging
There’s a lot happening behind the scenes to ensure that opening a Food Network result from Google Search is blazingly fast!

As you can see in the animation above, the final URL for the AMP content is on the foodnetwork.com domain, exactly as desired, and with the fast load speed that people enjoy with AMP pages. We’re very excited to validate that the imagined approach works across multiple layers of technology. However, it’s important to keep in mind that it’s still early and what you see isn’t ready to ship. As noted above, we expect more feedback from browser vendors and the web community to further refine the Web Packaging related specifications and work toward finalizing them.

Please reach out if you’re interested in this area and we’ll continue our commitment to provide updates as there is additional progress to share.

Posted by Kinuko Yasuda, Chrome Software Engineer, and Eric Steinlauf, Software Engineer, Google Search

A first look at using web packaging to improve AMP URLs

Join the AMP team at Google I/O 2018!

Starting Tuesday, May 8th, many members of the team will share the latest AMP updates and news at Google I/O 2018, Google’s annual developer conference. With many of the core AMP contributors working at Google, it gives us an opportunity to share the most recent work from the team.

This year there are many talks which include AMP across topics like e-commerce, WordPress, Progressive Web Apps and more. All of the talks will be live streamed on the I/O website and then added to the AMP YouTube Channel. For those of you who are able to join us in person, AMP also has demo tables at the Web Sandbox and Office Hours to answer your questions.

Talks that feature AMP content are listed below – be sure to check them out on live stream or watch the recordings on the AMP YouTube Channel afterwards!

Posted by Matt Ludwig, AMP Outreach/Marketing lead at Google

Join the AMP team at Google I/O 2018!

New functionality to help manage user choice in AMP pages

Over the past few weeks, we launched tools to help publishers collect user consent. Today, we’d like to give you an update on some new functionality  that may make it easier for publishers to collect consent on AMP pages.

Ad Network integrations with the <amp-consent> component

The DoubleClick and AdSense implementations of <amp-ad> now support <amp-consent>, including non-personalized ad serving. See detailed documentation about when and how to serve non-personalized ads from DoubleClick and AdSense and availability here.

Separately, if you are an ads or analytics vendor, you can learn more about how you can integrate with <amp-consent> here.

<amp-ima-video> support for <amp-consent>

Publishers can now collect consent when using the <amp-ima-video> component. See availability date here.

Using <amp-geo> for geolocation-based configuration of <amp-consent>

AMP provides a way to configure <amp-consent> with a remote call by using the checkConsentHref field. If you wished to incorporate geolocation-based logic for <amp-consent>, you  would need to resolve geolocation on the server-side, and customize the checkConsentHref response accordingly. Until now, it was not possible to have geolocation-based logic without a remote server setup.

The <amp-geo> component recently launched to enable content variation based on users’ approximate location, at the level of an ISO Country Code. Now, you can use this functionality in combination with <amp-consent>, which also means you could skip implementing checkConsentHref, if your configuration needs do not extend beyond geolocation. Checkout the ampbyexample.com  <amp-geo> sample or the <amp-consent> documentation.  

<amp-consent> support in AMP stories

Coming soon, <amp-consent> will be available to use with <amp-story>.  For the <amp-story> use case only, most of the UI will be pre-configured, though there will be publisher-specifiable choices for the look and feel in addition to full control over the text content. In addition, all <amp-consent> UIs in AMP stories will be blocking (i.e., the consent UI will not have an optional close button). See availability date here.

amp_consent

New features in <amp-consent>

checkConsentHref support for key-value pairs : This feature allows the checkConsentHref endpoint to respond with key-value pairs that can be propagated to and consumed by the ad network implementation. This is useful for publishers who want to communicate additional configuration information to ad networks to serve appropriate ads. This information is available to AMP components that wait on <amp-consent> to resolve. More details here.

Timeouts in <amp-consent> : With this feature, publishers could unblock components waiting for consent after a certain timeout. The feature also allows publishers to configure consent state that would be obtained by the AMP components blocked on the consent. More details here.

Availability

All this functionality is launching at different times in May, 18. Below is a summary of availability and you can go here for the latest updates.

Functionality Prod Availability Documentation Ready For Testing?
DoubleClick & AdSense 05/10/18 Link Yes
AMP IMA Video Integration 05/15/18 No
AMP Geo 05/10/18 Link Yes
AMP Stories 05/15/18 No

What’s next?

There are a number of features planned for <amp-consent> including supporting external consent flows and many others. If you have features in mind that haven’t been discussed yet, please add a comment to the master issue.

Also, don’t forget to check out the amp-consent usage examples on AMP By Example.

As always, thanks for your support. If you have any questions, let us know by opening an issue on GitHub.

Posted by Vamsee Jasti, Product Manager, AMP Project

New functionality to help manage user choice in AMP pages