包简介

A Laravel package to parse meta information from an URL.

README 文档

README

This package provides a simple helper to retrieve the HTML meta tags of a URL. It properly handles connection and client errors and converts the meta tag contents from the source encoding to UTF-8 if possible.

Installation & Usage

You can install this package via Composer:

composer require kovah/laravel-html-meta

Laravel automatically detects the package and makes it available in your application.

Usage

The HtmlMeta class is available as a facade and exposes the forUrl function. Here is a very basic example.

try {
    $metaTags = \Kovah\HtmlMeta\Facades\HtmlMeta::forUrl('https://kovah.de')->getMeta();
} catch (\Kovah\HtmlMeta\Exceptions\InvalidUrlException $e) {
    // the provided URL is invalid
} catch (\Kovah\HtmlMeta\Exceptions\DisallowedIpException $e) {
    // the URL points to a blocked non-public IP address
} catch (\Kovah\HtmlMeta\Exceptions\UnreachableUrlException $e) {
    // the website under this URL is not reachable
}

The $metaTags variable now contains the following data:

[
  "title" => "Kovah.de - Web Development and Photography",
  "generator" => "Hugo 0.58.2",
  "viewport" => "width=device-width, initial-scale=1",
  "description" => "Kovah - Web Development by Kevin Woblick",
  "og:title" => "Kovah.de - Web Development by Kevin Woblick",
  "og:description" => "Kovah - Web Development by Kevin Woblick",
  "og:image" => "'https://kovah.de/kvh_social_1200x630.jpg'",
  "og:url" => "'https://kovah.de/'/",
  "og:site_name" => "Portfolio of Kevin Woblick",
  "twitter:card" => "summary_large_image",
  // ...
];

If you want to use the response of the original request made to parse the HTML meta, you can get it with the getResponse() method like this:

$metaResults = \Kovah\HtmlMeta\Facades\HtmlMeta::forUrl('https://kovah.de');

$response = $metaResults->getResponse(); // Illuminate\Http\Client\Response
$metaTags = $metaResults->getMeta(); // array
$url = $metaResults->getUrl(); // string

Parsing HTML

As an alternative to fetching HTML from a URL, you might also parse HTML directly:

$myHtml = '<!DOCTYPE html><head><meta name=" twitter:description " content="Text Value for Twitter Description"> ...';
$metaTags = \Kovah\HtmlMeta\Facades\HtmlMeta::fromHtml($myHtml)->getMeta();

To improve parsing and fallbacks, you might pass HTTP headers and the URL to the fromHtml() method like this:

$myUrl = 'https://kovah.de';
$httpHeaders = [
    'content-type' => 'text/html; charset=iso-8859-1',
];
$myHtml = '<!DOCTYPE html><head><meta name=" twitter:description " content="Text Value for Twitter Description"> ...';
$metaTags = \Kovah\HtmlMeta\Facades\HtmlMeta::fromHtml($myHtml, $httpHeaders, $myUrl)->getMeta();

Configuration

By default, the package uses a 10 seconds timeout when trying to fetch the content of the URL. If you want to increase or decrease this timeout, you can publish the HTML Meta configuration.

php artisan vendor:publish --provider="Kovah\HtmlMeta\HtmlMetaServiceProvider"

The configuration can now be found under config/html-meta.php.

Setting a custom User Agent

The package allows you to set one or more custom User Agents which will be used to send the requests. The User Agent(s) you want to use must be specified as an array in the package configuration html-meta.php like this:

'user_agents' => [
    'Mozilla/5.0 (Windows NT 6.4) AppleWebKit/537.36.0 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36.0',
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12) AppleWebKit/…4.1.28 (KHTML, like Gecko) Version/15.2.0 Safari/604.1.28',
    'Mozilla/5.0 (compatible; Googlebot/2.1.0; +http://www.google.com/bot.html)',
]

The HTML Meta package will randomly choose one of the User Agents for each request. If you want to use only one User Agent, remove all others from the list:

'user_agents' => [
    'Mozilla/5.0 (Windows NT 6.4) AppleWebKit/537.36.0 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36.0'
]

Adding more custom headers

The custom_headers configuration can contain any headers that should be added to any request, except User-Agent and Accept. It can be either an array of headers, or a pipe-separated string.

'custom_headers' => [
    'Accept-Encoding' => 'gzip, deflate',
    'referer' => 'https://example.com',
],

If headers are passed as a string, the following format applies: [header name]=[value]|[header name]=[value]|...

Note: Pipes inside the headers as a string must be escaped with a backslash.

'custom_headers' => 'Accept-Encoding=gzip,deflate|referer=https://example.com'

Adding custom Guzzle client options

The custom_options configuration is an advanced feature and can be used to pass custom options to the Guzzle HTTP client. Possible options are documented here: http://docs.guzzlephp.org/en/stable/request-options.html

'custom_options' => [
    'allow_redirects' => false,
],

Blocking private IP targets

If you want to prevent requests to non-public IP addresses, enable the block_private_ips option:

'block_private_ips' => true,

When enabled, the package:

rejects URLs that already use a non-public IPv4 or IPv6 host directly
resolves hostname targets before the request is sent
throws Kovah\HtmlMeta\Exceptions\DisallowedIpException if any resolved DNS record points to a non-public IP range

This includes private, loopback, link-local and other reserved IP ranges.

Parsing Details

The default parser shipping with this package extracts the meta tags from the HTML. These are the steps it is going through after the packages received a successful response:

All meta tags with name or property properties are parsed from the <head> section. The keys are converted to lowercase.
The <title> tag is parsed and all excessive white space is removed from the start and the end of it.
The package checks for a charset, which can be specified as:
- the HTML charset meta tag (<meta charset="utf-8">),
- the HTTP content-type header (content-type: "text/html; charset=utf-8"),
- or as the HTML http-equiv="content-type" tag (<meta http-equiv="content-type" content="text/html; charset=utf-8">) We try to parse the charset in this exact order.
The value of all parsed meta tags is converted from the source charset (if available) to UTF-8, if it does not match UTF-8. If the meta tag value cannot be converted, it is replaced by null! The only exception is the title, which will be replaced by the hostname of the URL in case a conversion is not possible.
HTML entities such as — are converted to the correct characters, in this example —.

This package is a project by Kevin Woblick and Contributors

kovah/laravel-html-meta 适用场景与选型建议

kovah/laravel-html-meta 是一款基于 PHP 开发的 Composer 扩展包，目前已累计 55.35k 次下载、GitHub Stars 达 8，最近一次更新时间为 2021 年 03 月 07 日，在 PHP 生态内属于活跃度较高的组件。

它主要适用于以下技术方向：「parser」「html」「laravel」「meta」等业务场景。在实际项目中，围绕这些方向常见需要落地的问题包括：接口对接、性能调优、并发安全、与既有框架（Laravel / ThinkPHP / Yii / Webman 等）的兼容适配，以及生产环境的日志埋点与稳定性保障。

我们在过去多个企业项目中使用过 kovah/laravel-html-meta 或与其功能相近的方案，如果你在选型或落地过程中遇到问题，例如 版本兼容、二次改造、私有化封装、与内部系统对接、生产 BUG 排查，欢迎联系我们协助评估。

围绕 kovah/laravel-html-meta 我们能提供哪些服务？

定制开发 / 二次开发

基于 kovah/laravel-html-meta 在你已有业务上做功能扩展、字段裁剪、UI 适配、与内部账号 / 权限 / 日志系统的深度对接。

BUG 修复 & 性能优化

线上偶发问题、内存泄漏、慢查询、并发异常等排查修复；针对高流量场景做缓存、队列、索引层面的调优。

项目外包 & 长期维护

承接完整的项目从需求 → 设计 → 开发 → 上线 → 长期运维；也可按月提供技术保姆服务。

yvsm@zunyunkeji.com QQ：316430983 微信：yvsm316 西安尊云信息科技 · 专注 PHP / Go / 分布式系统研发

与 kovah/laravel-html-meta 相关的其它包

同方向 / 同关键字的高下载量 PHP Composer 包推荐，方便对比选型：

dg/texy 159

Texy converts plain text in easy to read Texy syntax into structurally valid (X)HTML. It supports adding of images, links, nested lists, tables and has full support for CSS. Texy supports hyphenation of long words (which reflects language rules), clickable emails and URL (emails are obfuscated again

chojnicki/video-platforms-parser 222

Easy to use SDK with grabber for multiple platforms at once like YouTube, Dailymotion, Facebook and more.

marketpay/mt940 0

An MT940 bank statement parser for PHP

axelarge/html-helpers 6

HTML and form generation

drpdigital/laravel-json-api-parser 3

Laravel integration for the jsonapi.org parser

eyupcanakman/simple-html-dom 1

A modern HTML DOM parser for PHP using DOMDocument and Symfony CssSelector.