Scala Http 请求调用 － scalaj-http

Simplified Http

This is a fully featured http client for Scala which wraps java.net.HttpURLConnection

Features:

• Zero dependencies
• Cross compiled for Scala 2.10, 2.11, 2.12, and 2.13-M3
• OAuth v1 request signing
• Automatic support of gzip and deflate encodings from server
• Easy to add querystring or form params. URL encoding is handled for you.
• Multipart file uploads

Non-Features:

• Async execution
  • The library is thread safe. HttpRequest and HttpResponse are immutable. So it should be easy to wrap in an execution framework of your choice.

Works in Google AppEngine and Android environments.

Note: 2.x.x is a new major version which is both syntactically and behaviorally different than the 0.x.x version.

Previous version is branched here: https://github.com/scalaj/scalaj-http/tree/0.3.x

Big differences:

• Executing the request always returns a HttpResponse[T] instance that contains the response-code, headers, and body
• Exceptions are no longer thrown for 4xx and 5xx response codes. Yay!
• Http(url) is the starting point for every type of request (post, get, multi, etc)
• You can easily create your own singleton instance to set your own defaults (timeouts, proxies, etc)
• Sends "Accept-Encoding: gzip,deflate" request header and decompresses based on Content-Encoding (configurable)
• Redirects are no longer followed by default. Use .option(HttpOptions.followRedirects(true)) to change.

Installation

in your build.sbt

libraryDependencies +=  "org.scalaj" %% "scalaj-http" % "2.4.1"

　　

maven

<dependency>
  <groupId>org.scalaj</groupId>
  <artifactId>scalaj-http_${scala.version}</artifactId>
  <version>2.4.1</version>
</dependency>

　　

If you're including this in some other public library. Do your users a favor and change the fully qualified name so they don't have version conflicts if they're using a different version of this library. The easiest way to do that is just to copy the source into your project :)

Usage

Simple Get

import scalaj.http._

val response: HttpResponse[String] = Http("http://foo.com/search").param("q","monkeys").asString
response.body
response.code
response.headers
response.cookies

Immutable Request

Http(url) is just shorthand for a Http.apply which returns an immutable instance of HttpRequest.
You can create a HttpRequest and reuse it:

val request: HttpRequest = Http("http://date.jsontest.com/")

val responseOne = request.asString
val responseTwo = request.asString

Additive Request

All the "modification" methods of a HttpRequest are actually returning a new instance. The param(s), option(s), header(s) methods always add to their respective sets. So calling .headers(newHeaders) will return a HttpRequest instance that has newHeaders appended to the previous req.headers

Simple form encoded POST

Http("http://foo.com/add").postForm(Seq("name" -> "jon", "age" -> "29")).asString

OAuth v1 Dance and Request

Note: the .oauth(...) call must be the last method called in the request construction

import scalaj.http.{Http, Token}

val consumer = Token("key", "secret")
val response = Http("https://api.twitter.com/oauth/request_token").postForm(Seq("oauth_callback" -> "oob"))
  .oauth(consumer).asToken

println("Go to https://api.twitter.com/oauth/authorize?oauth_token=" + response.body.key)

val verifier = Console.readLine("Enter verifier: ").trim

val accessToken = Http("https://api.twitter.com/oauth/access_token").postForm.
  .oauth(consumer, response.body, verifier).asToken

println(Http("https://api.twitter.com/1.1/account/settings.json").oauth(consumer, accessToken.body).asString)

Parsing the response

Http("http://foo.com").{asString, asBytes, asParams}

Those methods will return an HttpResponse[String | Array[Byte] | Seq[(String, String)]] respectively

Advanced Usage Examples

Parse the response InputStream directly

val response: HttpResponse[Map[String,String]] = Http("http://foo.com").execute(parser = {inputStream =>
  Json.parse[Map[String,String]](inputStream)
})

Post raw Array[Byte] or String data and get response code

Http(url).postData(data).header("content-type", "application/json").asString.code

Post multipart/form-data

Http(url).postMulti(MultiPart("photo", "headshot.png", "image/png", fileBytes)).asString

You can also stream uploads and get a callback on progress:

Http(url).postMulti(MultiPart("photo", "headshot.png", "image/png", inputStream, bytesInStream,
  lenWritten => {
    println(s"Wrote $lenWritten bytes out of $bytesInStream total for headshot.png")
  })).asString

Stream a chunked transfer response (like an event stream)

Http("http://httpbin.org/stream/20").execute(is => {
  scala.io.Source.fromInputStream(is).getLines().foreach(println)
})

note that you may have to wrap in a while loop and set a long readTimeout to stay connected

Send https request to site with self-signed or otherwise shady certificate

Http("https://localhost/").option(HttpOptions.allowUnsafeSSL).asString

Do a HEAD request

Http(url).method("HEAD").asString

Custom connect and read timeouts

These are set to 1000 and 5000 milliseconds respectively by default

Http(url).timeout(connTimeoutMs = 1000, readTimeoutMs = 5000).asString

Get request via a proxy

val response = Http(url).proxy(proxyHost, proxyPort).asString

Other custom options

The .option() method takes a function of type HttpURLConnection => Unit so you can manipulate the connection in whatever way you want before the request executes.

Change the Charset

By default, the charset for all param encoding and string response parsing is UTF-8. You can override with charset of your choice:

Http(url).charset("ISO-8859-1").asString

Create your own HttpRequest builder

You don't have to use the default Http singleton. Create your own:

object MyHttp extends BaseHttp (
  proxyConfig = None,
  options = HttpConstants.defaultOptions,
  charset = HttpConstants.utf8,
  sendBufferSize = 4096,
  userAgent = "scalaj-http/1.0",
  compress = true
)

Full API documentation

scaladocs here

Dealing with annoying java library issues

Overriding the Access-Control, Content-Length, Content-Transfer-Encoding, Host, Keep-Alive, Origin, Trailer, Transfer-Encoding, Upgrade, Via headers

Some of the headers are locked by the java library for "security" reasons and the behavior is that the library will just silently fail to set them. You can workaround by doing one of the following:

• Start your JVM with this command line parameter: -Dsun.net.http.allowRestrictedHeaders=true
• or, do this first thing at runtime: System.setProperty("sun.net.http.allowRestrictedHeaders", "true")

以上官方说明，下面具体怎么使用：

首先Http请求，返回json类型数据，就要有json解析及转化工具。

Scala　sbt引入scalaj-http及spray-json.

"org.scalaj" % "scalaj-http_2.12" % "2.3.0",
"com.github.wangzaixiang" %% "spray-json" % "1.3.4",
"com.google.code.gson" % "gson" % "2.7"

　　

工具类如下：

package test.scalajhttp

import com.google.gson.Gson
import scalaj.http.{Http, HttpOptions, HttpResponse}
import spray.json.DefaultJsonProtocol._
import spray.json.{JsValue, _}

/**
  * 类功能描述：Scalaj http 工具类
  *
  * @author WangXueXing create at 19-8-15 下午5:56
  * @version 1.0.0
  */
object ScalajHttpUtil {
  /**
    * 成功
    */
  val STATUS_SUCCESS: String = "success"
  /**
    * 失败
    */
  val STATUS_FAIL: String = "fail"

  /**
    * http公共请求类
    *
    * @param url  Http路径如：https://www.baidu.com
    * @param params 参数
    * @return
    */
  def requestUrl(url: String, params: Map[String, String]) = {
    val httpRequest = Http(url)
                      .header("Content-Type", "application/xml")//"application/json"
                      .header("Charset", "UTF-8")
                      .option(HttpOptions.readTimeout(10000))
    val response: HttpResponse[String] = (params != null && !params.isEmpty) match {
        case true =>  httpRequest.params(params).asString
        case false => httpRequest.asString
    }

    response.body
  }

  /**
    * http公共请求类
    *
    * @param url  Http路径如：https://www.baidu.com
    * @param requestJson Json参数
    * @return
    */
  def postRequestUrl(url: String, requestJson: String) = {
    val httpRequest = Http(url)
      .header("Content-Type", "application/xml")//"application/json"
      .header("Charset", "UTF-8")
      .option(HttpOptions.readTimeout(10000))
    val response: HttpResponse[String] = (requestJson != null && !requestJson.isEmpty) match {
      case true =>  httpRequest.postData(requestJson).asString
      case false => httpRequest.asString
    }

    response.body
  }

  /**
    * 将object映射json string
    *
    * @param obj
    * @return
    */
  def objToJson(obj: NcRespose): String = {
    new Gson().toJson(obj)
  }

  /**
    * 将map数据映射到Object
    *
    * @param jsonStr
    * @return
    */
  def jsonToObject(jsonStr: String): NcRespose = {
    val jsonObj = jsonStr.parseJson.convertTo[Map[String, JsValue]]
    mapToObject(jsonObj)
  }

  /**
    * 将map数据映射到PRackType类上
    *
    * @param jsonMap
    * @return
    */
  def mapToJson(jsonMap: Map[String, JsValue]): String = {
    val obj = mapToObject(jsonMap)
    new Gson().toJson(obj)
  }

  /**
    * 将map数据映射到Object
    *
    * @param jsonObj
    * @return
    */
  def mapToObject(jsonObj: Map[String, JsValue]): NcRespose = {
    NcRespose(
      status = toStringValue(jsonObj.get("status").get),
      message = toStringValue(jsonObj.get("message").get)
    )
  }

  def toStringValue(jsValue: JsValue): String = {
    s"$jsValue".replace("\"", "")
  }

  def toIntValue(jsValue: JsValue): Int = {
    s"$jsValue".replace("\"", "").toInt
  }

  def toLongValue(jsValue: JsValue): Long = {
    s"$jsValue".replace("\"", "").toLong
  }

  def main(args: Array[String]): Unit = {
    val jsonStr1 = "{\"billhead\":{\"bill_date\":\"2019-08-13\",\"creationtime\":\"2019-08-13 17:55:00\",\"creator\":\"1000\",\"local_money\":100,\"local_rate\":1,\"objecttype\":0,\"payway\":0,\"pk_currtype\":\"CNY\",\"pk_org\":\"101\",\"primal_money\":100},\"items\":[{\"billdetail_no\":0,\"pay_local\":100,\"pay_primal\":100,\"pk_oppaccount\":\"421860158018800081218\"}]}"
    val jsonStr = postRequestUrl("http://10.100.99.98:8000/uapws/rest/SyncCMPPayBillRestful/add", jsonStr1)

    val gson = new Gson()
    val respose = gson.fromJson(jsonStr, classOf[NcRespose])

//    val respose = jsonToObject(jsonStr)
//    println(jsonStr)
    if(respose.status == STATUS_SUCCESS){
      println(s"success: ${respose.message}")
    } else {
      println(s"fail: ${respose.message}")
    }
  }
}

case class NcRespose(status: String, message: String)

scala中使用json，比java中使用json复杂一些。尝试了json-smart；fast json； gson；json4s以及scala自带的json。

总结如下：

1. 普通json string 转化 object, 使用JSON.pareseFull 。

JSON.parseFull(str).get.asInstanceOf[Map[String, String]]("key") 

2. json 与 map 互相转化

val colors:Map[String,Object] = Map("red" -> "123456", "azure" -> "789789")
val json = JSONObject(colors)
println(json)
val jsonMap = JSON.parseFull(json.toString).get.asInstanceOf[Map[String,Object]]
println(jsonMap) 

3. json 与 class 互相转化

case class Student( name:String , no: String )

    val gson = new Gson
    val student = Student("张三", "100")
    val str = gson.toJson(student, classOf[Student])
    println(str)
    val student2 = gson.fromJson(str, classOf[Student])
    println(student2)

尤其注意：使用 classOf ，不能使用 getclass

4. json与Map互相转化，需使用java的map

val map = new util.HashMap[String, Object]()
map.put("abc", List(s1,s2).toArray)
val gson = new Gson()
println( gson.toJson(map) )
另附，json判断代码

def isGoodJson(json: String):Boolean = {

if(null == json) {
return false
}
val result = JSON.parseFull(json) match {
case Some(_: Map[String, Any]) => true
case None => false
case _ => false
}
result
}

请参考：

scalaj-http