jax-rs jax-ws

有時，有必要通過JPA檢索大型數據集（例如，超過1,000,000條記錄），并將它們填充到java.util.List的單個實例中是有風險的（內存障礙）。 因此，這是一個快速的解決方案，它可以解決JAX-RS REST資源端點如何仍能通過“頁面”對JPA實體進行流式處理或序列化而在不破壞內存約束的情況下及時提供響應。

示例數據庫表和JPA實體

數據庫表

為了演示如何實現大數據輸出，這是我們可以使用的示例MySQL數據庫表。

create database large_data_test_db; use large_data_test_db;create table generated_uuids (record_no bigint not null auto_increment,uuid varchar(100) not null,datetime_generated datetime not null,primary key(record_no),unique(uuid) );

JPA實體

接下來，定義代表上述表結構的JPA實體類。

GeneratedUuidEntity.java的代碼

package com.developerscrappad;import java.io.Serializable; import java.util.Date; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.NamedQueries; import javax.persistence.NamedQuery; import javax.persistence.Table; import javax.persistence.Temporal; import javax.persistence.TemporalType;@Entity @Table( name = "generated_uuids" ) @NamedQueries( {@NamedQuery( name = "GeneratedUuidEntity.listAll", query = "SELECT u FROM GeneratedUuidEntity u" ),@NamedQuery( name = "GeneratedUuidEntity.queryRecordsSize", query = "SELECT count(u) FROM GeneratedUuidEntity u" ) } ) public class GeneratedUuidEntity implements Serializable {private static final long serialVersionUID = 12312312234234123L;@Id@GeneratedValue( strategy = GenerationType.IDENTITY )@Column( name = "record_no" )private Long recordNo;@Column( name = "uuid" )private String uuid;@Column( name = "datetime_generated" )@Temporal( TemporalType.TIMESTAMP )private Date datetimeGenerated;public GeneratedUuidEntity() {}public GeneratedUuidEntity( Long recordNo ) {this.recordNo = recordNo;}public GeneratedUuidEntity( Long recordNo, String uuid, Date datetimeGenerated ) {this.recordNo = recordNo;this.uuid = uuid;this.datetimeGenerated = datetimeGenerated;}public Long getRecordNo() {return recordNo;}public void setRecordNo( Long recordNo ) {this.recordNo = recordNo;}public String getUuid() {return uuid;}public void setUuid( String uuid ) {this.uuid = uuid;}public Date getDatetimeGenerated() {return datetimeGenerated;}public void setDatetimeGenerated( Date datetimeGenerated ) {this.datetimeGenerated = datetimeGenerated;}@Overridepublic int hashCode() {int hash = 0;hash += ( recordNo != null ? recordNo.hashCode() : 0 );return hash;}@Overridepublic boolean equals( Object object ) {// TODO: Warning - this method won't work in the case the id fields are not setif ( !( object instanceof GeneratedUuidEntity ) ) {return false;}GeneratedUuidEntity other = ( GeneratedUuidEntity ) object;if ( ( this.recordNo == null && other.recordNo != null ) || ( this.recordNo != null && !this.recordNo.equals( other.recordNo ) ) ) {return false;}return true;}@Overridepublic String toString() {return "com.developerscrappad.GeneratedUuidEntity[ recordNo=" + recordNo + " ]";} }

在GeneratedUuidEntity中定義了兩個命名查詢。 GeneratedUuidEntity.queryRecordsSize用于查詢表的總記錄數，而GeneratedUuidEntity.listAll用于檢索表中的所有記錄。

實施JAX-RS REST資源（Java EE方式）

讓我們有一個名稱為JPAStreamingRESTResource的JAX-RS REST資源類，其中有一個可用的JPA EntityManager（持久性名稱： JPAStreamingPU ）要注入，并通過受保護的方法getEntityManager（）獲得 。

@Path( "generated-uuids" ) @Stateless( name = "JPAStreamingRESTResource", mappedName = "ejb/JPAStreamingRESTResource" ) public class JPAStreamingRESTResource {@PersistenceContext( unitName = "JPAStreamingPU" )private EntityManager entityManager;protected EntityManager getEntityManager() {return entityManager;}/*** Say "NO" to response caching*/protected Response.ResponseBuilder getNoCacheResponseBuilder( Response.Status status ) {CacheControl cc = new CacheControl();cc.setNoCache( true );cc.setMaxAge( -1 );cc.setMustRevalidate( true );return Response.status( status ).cacheControl( cc );} }

此外，我們有一個名為getNoCacheResponseBuilder（）的方法，該方法將獲得一個非緩存的javax.ws.rs.core.Response.ResponseBuilder ，這樣以后就不會再得到奇怪的緩存結果了。

JPA調用方法

接下來，讓我們在資源類中定義兩個方法，即：

queryGeneratedUuidRecordsSize（） –檢索表中的記錄總數

private int queryGeneratedUuidRecordsSize() {return getEntityManager().createNamedQuery( "GeneratedUuidEntity.queryRecordsSize", Long.class ).getSingleResult().intValue(); }

listAllGeneratedUuidEntities（） –從表中檢索所有數據，但具有某些限制條件，例如記錄的開始位置（recordPosition）和每次往返數據庫的最大記錄數（recordsPerRoundTrip）。 目的是“分頁”結果，以使結果列表不會過分膨脹。 我們稍后會看到它的作用。

private List<GeneratedUuidEntity> listAllGeneratedUuidEntities( int recordPosition, int recordsPerRoundTrip ) {return getEntityManager().createNamedQuery( "GeneratedUuidEntity.listAll" ).setFirstResult( recordPosition ).setMaxResults( recordsPerRoundTrip ).getResultList(); }

讓流開始

現在，讓我們實現資源端點方法，至少從理論上講，該方法可以在不損害大小的情況下檢索數據。 此方法將返回JSON響應，其數據格式為：

{"result": [{"record_no": 1,"uuid": "34d99089-3e36-4f00-ab93-846b61771eb3","datetime_generated": "2015-06-28 21:02:23"},…] }@GET@Path( "list-all" )@Produces( "application/json" )@TransactionAttribute( TransactionAttributeType.NEVER )public Response streamGeneratedUuids() {// Define the format of timestamp outputSimpleDateFormat df = new SimpleDateFormat( "yyyy-MM-dd HH:mm:ss" );return getNoCacheResponseBuilder( Response.Status.OK ).entity( new StreamingOutput() {// Instruct how StreamingOutput's write method is to stream the data@Overridepublic void write( OutputStream os ) throws IOException, WebApplicationException {int recordsPerRoundTrip = 100; // Number of records for every round trip to the databaseint recordPosition = 0; // Initial record position indexint recordSize = queryGeneratedUuidRecordsSize(); // Total records found for the query// Start streaming the datatry ( PrintWriter writer = new PrintWriter( new BufferedWriter( new OutputStreamWriter( os ) ) ) ) {writer.print( "{\"result\": [" );while ( recordSize > 0 ) {// Get the paged data set from the DBList<GeneratedUuidEntity> generatedUuidEntities = listAllGeneratedUuidEntities( recordPosition, recordsPerRoundTrip );for ( GeneratedUuidEntity generatedUuidEntity : generatedUuidEntities ) {if ( recordPosition > 0 ) {writer.print( "," );}// Stream the data in Json object formatwriter.print( Json.createObjectBuilder().add( "record_no", generatedUuidEntity.getRecordNo() ).add( "uuid", generatedUuidEntity.getUuid() ).add( "datetime_generated", df.format( generatedUuidEntity.getDatetimeGenerated() ) ).build().toString() );// Increase the recordPosition for every record streamedrecordPosition++;}// update the recordSize (remaining no. of records)recordSize -= recordsPerRoundTrip;}// Done!writer.print( "]}" );}}} ).build();}

電源線說明：

實際上，這很簡單。 訣竅是通過重寫write（）方法來定義匿名類StreamingOutput的表達式，該方法首先通過queryGeneratedUuidRecordsSize（）查詢總記錄大小，然后通過listAllGeneratedUuidEntities（）逐頁檢索記錄。 此方法將多次訪問數據庫，具體取決于定義的recordsPerRoundTrip值。

JPAStreamingRESTResource.java的完整源代碼：

package com.developerscrappad;import java.io.BufferedWriter; import java.io.IOException; import java.io.OutputStream; import java.io.OutputStreamWriter; import java.io.PrintWriter; import java.text.SimpleDateFormat; import java.util.List; import javax.ejb.Stateless; import javax.ejb.TransactionAttribute; import javax.ejb.TransactionAttributeType; import javax.json.Json; import javax.persistence.EntityManager; import javax.persistence.PersistenceContext; import javax.ws.rs.GET; import javax.ws.rs.Path; import javax.ws.rs.Produces; import javax.ws.rs.WebApplicationException; import javax.ws.rs.core.CacheControl; import javax.ws.rs.core.Response; import javax.ws.rs.core.StreamingOutput;@Path( "generated-uuids" ) @Stateless( name = "JPAStreamingRESTResource", mappedName = "ejb/JPAStreamingRESTResource" ) public class JPAStreamingRESTResource {@PersistenceContext( unitName = "JPAStreamingPU" )private EntityManager entityManager;private List<GeneratedUuidEntity> listAllGeneratedUuidEntities( int recordPosition, int recordsPerRoundTrip ) {return getEntityManager().createNamedQuery( "GeneratedUuidEntity.listAll" ).setFirstResult( recordPosition ).setMaxResults( recordsPerRoundTrip ).getResultList();}private int queryGeneratedUuidRecordsSize() {return getEntityManager().createNamedQuery( "GeneratedUuidEntity.queryRecordsSize", Long.class ).getSingleResult().intValue();}protected EntityManager getEntityManager() {return entityManager;}/*** Say "NO" to response caching*/protected Response.ResponseBuilder getNoCacheResponseBuilder( Response.Status status ) {CacheControl cc = new CacheControl();cc.setNoCache( true );cc.setMaxAge( -1 );cc.setMustRevalidate( true );return Response.status( status ).cacheControl( cc );}@GET@Path( "list-all" )@Produces( "application/json" )@TransactionAttribute( TransactionAttributeType.NEVER )public Response streamGeneratedUuids() {// Define the format of timestamp outputSimpleDateFormat df = new SimpleDateFormat( "yyyy-MM-dd HH:mm:ss" );return getNoCacheResponseBuilder( Response.Status.OK ).entity( new StreamingOutput() {// Instruct how StreamingOutput's write method is to stream the data@Overridepublic void write( OutputStream os ) throws IOException, WebApplicationException {int recordsPerRoundTrip = 100; // Number of records for every round trip to the databaseint recordPosition = 0; // Initial record position indexint recordSize = queryGeneratedUuidRecordsSize(); // Total records found for the query// Start streaming the datatry ( PrintWriter writer = new PrintWriter( new BufferedWriter( new OutputStreamWriter( os ) ) ) ) {writer.print( "{\"result\": [" );while ( recordSize > 0 ) {// Get the paged data set from the DBList<GeneratedUuidEntity> generatedUuidEntities = listAllGeneratedUuidEntities( recordPosition, recordsPerRoundTrip );for ( GeneratedUuidEntity generatedUuidEntity : generatedUuidEntities ) {if ( recordPosition > 0 ) {writer.print( "," );}// Stream the data in Json object formatwriter.print( Json.createObjectBuilder().add( "record_no", generatedUuidEntity.getRecordNo() ).add( "uuid", generatedUuidEntity.getUuid() ).add( "datetime_generated", df.format( generatedUuidEntity.getDatetimeGenerated() ) ).build().toString() );// Increase the recordPosition for every record streamedrecordPosition++;}// update the recordSize (remaining no. of records)recordSize -= recordsPerRoundTrip;}// Done!writer.print( "]}" );}}} ).build();} }

小心

請記住要調整應用程序服務器的響應連接超時值，以防止REST或Http Client拋出java.io.IOException過早的EOF異常。

測試它

要測試它是否有效，只需在表中加載567條記錄即可。 然后，讓單元測試調用端點URL，并使用以下單元測試代碼將檢索到的JSON數據保存到文件中（使用Apache HttpClient）：

JPAStreamingUnitTest.java的代碼：

package com.developerscrappad;import java.io.File; import java.io.FileInputStream; import static org.junit.Assert.*;import java.nio.file.FileSystems; import java.nio.file.Files; import java.sql.Connection; import java.sql.DriverManager; import java.sql.PreparedStatement; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.UUID; import javax.json.Json; import javax.json.JsonArray; import javax.json.JsonObject; import javax.json.JsonReader; import org.apache.http.HttpEntity; import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.junit.AfterClass; import org.junit.BeforeClass; import org.junit.Test;public class JPAStreamingUnitTest {private static final String dbDriverClassname = "com.mysql.jdbc.Driver";private static final String dbUrl = "jdbc:mysql://localhost:3306/large_data_test_db";private static final String username = "username";private static final String password = "password";private static final int numberOfRecords = 567;private static final String jsonResultOutputFilename = "testing123.json";@BeforeClasspublic static void setUpClass() {try {Class.forName( dbDriverClassname );try ( Connection conn = DriverManager.getConnection( dbUrl, username, password ) ) {String insertSQL = "insert into generated_uuids (uuid, datetime_generated) values (?, now())";try ( PreparedStatement stmt = conn.prepareStatement( insertSQL ) ) {for ( int i = 0; i < numberOfRecords; i++ ) {System.out.println( "Inserting row: " + i );stmt.setString( 1, UUID.randomUUID().toString() );stmt.executeUpdate();}}}} catch ( final Exception ex ) {ex.printStackTrace();fail( ex.getMessage() );}}@AfterClasspublic static void tearDownClass() {try {Class.forName( dbDriverClassname );try ( Connection conn = DriverManager.getConnection( dbUrl, username, password ) ) {String truncateSQL = "truncate generated_uuids";conn.createStatement().executeUpdate( truncateSQL );}new File( System.getProperty( "java.io.tmpdir" ), jsonResultOutputFilename ).delete();} catch ( final Exception ex ) {ex.printStackTrace();fail( ex.getMessage() );}}@Testpublic void testJPAStreaming() {String url = "http://localhost:8080/JPAStreaming/rest-api/generated-uuids/list-all/";try {CloseableHttpClient httpclient = HttpClients.createDefault();HttpGet httpGet = new HttpGet( url );try ( CloseableHttpResponse response1 = httpclient.execute( httpGet ) ) {System.out.println( response1.getStatusLine() );HttpEntity entity1 = response1.getEntity();Files.copy( entity1.getContent(), FileSystems.getDefault().getPath( System.getProperty( "java.io.tmpdir" ), jsonResultOutputFilename ) );}// Validatetry ( JsonReader jsonReader = Json.createReader( new FileInputStream( new File( System.getProperty( "java.io.tmpdir" ), jsonResultOutputFilename ) ) ) ) {JsonObject jsonObj = jsonReader.readObject();assertTrue( jsonObj.containsKey( "result" ) );JsonArray jsonArray = jsonObj.getJsonArray( "result" );assertEquals( numberOfRecords, jsonArray.size() );SimpleDateFormat validationDF = new SimpleDateFormat( "yyyy-MM-dd HH:mm:ss" );for ( int i = 0; i < jsonArray.size(); i++ ) {JsonObject generatedUuidJsonObj = jsonArray.getJsonObject( i );int recordNumber = generatedUuidJsonObj.getInt( "record_no" );assertTrue( recordNumber > 0 );try {UUID.fromString( generatedUuidJsonObj.getString( "uuid" ) );} catch ( IllegalArgumentException ex ) {fail( "Invalid UUID format at record number: " + recordNumber );}try {validationDF.parse( generatedUuidJsonObj.getString( "datetime_generated" ) );} catch ( final NullPointerException | ParseException ex ) {fail( "datetime_generated field must not be null and must be of format yyyy-MM-dd HH:mm:ss" );}}}} catch ( final Exception ex ) {ex.printStackTrace();fail( ex.getMessage() );}} }

我們完成了。 感謝您的閱讀，希望對您有所幫助。

翻譯自: https://www.javacodegeeks.com/2015/07/how-to-streamserialize-jpa-result-as-jax-rs-response-for-large-data.html

jax-rs jax-ws

總結

以上是生活随笔為你收集整理的jax-rs jax-ws_如何以大数据的JAX-RS响应的形式将JPA结果流化/序列化的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。