鐚垮垱寰佹枃锝滃妯℃€佸浘鍍忓悎鎴愬拰缂栬緫锛圡ISE锛夛細Multimodal Image Synthesis and Editing: A Survey

鐢变簬淇℃伅鍦ㄧ幇瀹炰笘鐣屼腑浠ュ悇绉嶅舰寮忓瓨鍦?#xff0c;澶氭ā鎬佷俊鎭箣闂寸殑鏈夋晥浜や簰鍜岃瀺鍚堝浜庤绠楁満瑙嗚鍜屾繁搴﹀涔犵爺绌朵腑澶氭ā鎬佹暟鎹殑鍒涘缓鍜屾劅鐭ヨ捣鐫€鍏抽敭浣滅敤銆?/p>

杩戞湡 OpenAI 鍙戝竷鐨?DALLE-2 鍜岃胺姝屽彂甯冪殑 Imagen 绛夊疄鐜颁簡浠や汉鎯婂徆鐨勬枃瀛楀埌鍥惧儚鐨勭敓鎴愭晥鏋?#xff0c;寮曞彂浜嗗箍娉涘叧娉ㄥ苟涓旇鐢熷嚭浜嗗緢澶氭湁瓒g殑搴旂敤銆傝€?strong>鏂囧瓧鍒板浘鍍忕殑鐢熸垚灞炰簬澶氭ā鎬佸浘鍍忓悎鎴愪笌缂栬緫棰嗗煙鐨勪竴涓吀鍨嬩换鍔?/strong>銆傚妯℃€佸浘鍍忓悎鎴愬拰缂栬緫鍦ㄥ缓妯″妯℃€佷俊鎭箣闂寸殑浜や簰鏂归潰鍏锋湁寮哄ぇ鐨勮兘鍔?#xff0c;杩戝勾鏉ュ凡鎴愪负涓€涓儹闂ㄧ殑鐮旂┒璇鹃銆?/p>

鏈瘒鏂囩珷鏄槄璇籑ultimodal Image Synthesis and Editing: A Survey鐨勭簿璇荤瑪璁?#xff0c;璁烘枃鍙戣〃浜?021骞?2鏈?#xff0c;鏄竴绡囧€煎緱涓€璇荤殑缁艰堪銆?br /> 璁烘枃鍦板潃:https://arxiv.org/abs/2112.13592v1
椤圭洰鍦板潃:https://github.com/fnzhan/MISE.

Java毕设项目实战

数据安全

鐩綍

代码覆盖率

涓€銆佷粙缁?/h1>

浜虹被鑳藉鑷劧鑰岀劧鍦拌兘鏍规嵁鏂囨湰鎴栭煶棰戞弿杩板鍦烘櫙杩涜鎴愬儚銆傜劧鑰?#xff0c;瀵逛簬璁$畻鏈鸿€岃█,鍥犱负瑙嗚绾跨储鍜岀湡瀹炲浘鍍忎箣闂寸殑妯℃€佸唴闂撮殭鍜岄潪瑙嗚绾跨储鍜屽疄闄呭浘鍍忎箣闂寸殑璺ㄦā鎬侀棿闅?#xff0c;浜哄伐鏅鸿兘瀛樺湪瑙嗚鎰熺煡涓婄殑妯℃€侀殧闃?/strong>銆備粠涓嶅悓妯″紡鐨勬暟鎹腑鏈夋晥鍦版绱㈠拰铻嶅悎寮傛瀯淇℃伅浠嶇劧鏄浘鍍忓悎鎴愬拰缂栬緫涓殑涓€澶ф寫鎴樸€?/p>

澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫(MISE)鐨勪换鍔℃棬鍦ㄦā鎷熺幇瀹炰笘鐣屼腑鐨勪汉绫绘兂璞″姏鍜屽垱閫犲姏,涓烘繁鍏ヤ簡瑙g缁忕綉缁滃浣曞皢澶氭ā鎬佷俊鎭笌鍥惧儚灞炴€х浉鍏宠仈鎻愪緵浜嗘繁鍒荤殑瑙佽В銆傚浘鍍忓悎鎴愬拰缂栬緫鏃ㄥ湪鍒涘缓鐪熷疄鍥惧儚鎴栫紪杈戝叿鏈夎嚜鐒剁汗鐞嗙殑鐪熷疄鍥惧儚銆?/p>

浜屻€佽法妯℃€佸紩瀵?/h1>

淇℃伅鐨勬瘡涓€绉嶆潵婧愭垨褰㈠紡閮藉彲浠ョО涓轰竴绉嶆ā鎬?/strong>銆備緥濡?#xff0c;浜轰滑鏈夎Е瑙夈€佸惉瑙夈€佽瑙夊拰鍡呰;淇℃伅濯掍粙鍖呮嫭璇煶銆佽棰戙€佹枃鏈瓑,浠ュ強鐢卞悇绉嶄紶鎰熷櫒(渚嬪闆疯揪銆佺孩澶栧拰鍔犻€熷害璁?#xff09;璁板綍鐨勬暟鎹€備笂杩版瘡涓暟鎹舰寮忛兘鍙互绉颁负涓€绉嶆ā鎬?#xff08;鎴栨ā寮?#xff09;銆傚湪鍥惧儚鍚堟垚鍜岀紪杈戞柟闈?#xff0c;鎴戜滑灏嗗紩瀵兼ā鎬佸垎涓鸿瑙夊紩瀵笺€佹枃鏈紩瀵笺€侀煶棰戝紩瀵煎拰鍏朵粬妯℃€併€備互涓嬪皬鑺傚皢璇︾粏鎻忚堪姣忕妯℃€佷互鍙婄浉鍏冲鐞嗘柟娉曘€?/p>

2.1銆佽瑙夊紩瀵?/h2>

瑙嗚寮曞鍖呮嫭浠庝竴浜涜瑙夌嚎绱㈠鍒嗗壊鍥俱€佸叧閿偣銆佹覆鏌撳嚑浣曘€佽竟缂樺浘銆佸満鏅竷灞€鍥俱€侀紶鏍囪建杩广€佸厜绾垮浘绛夌瓑瑙嗚寮曞鐢熸垚鐪熷疄鍥惧儚,鍥犱负瑙嗚绾跨储鍙互琚涓烘煇绉嶇被鍨嬬殑鍥惧儚,鍏跺厑璁镐娇鐢ㄥ嵎绉眰鐩存帴缂栫爜浠ヤ骇鐢熺洰鏍囩敓鎴愭垨缂栬緫銆傜敱浜庤瑙変俊鎭腑鐨勭簿纭拰娓呮櫚寮曞,瑙嗚寮曞鍙互涓庡浘鍍忓悎鎴愪腑鐨勭湡瀹炲浘鍍忛厤瀵规垨涓嶉厤瀵广€?br />

篮球小程序

閫氳繃缂栬緫瑙嗚寮曞,渚嬪璇箟鍥?#xff0c;鍥惧儚鍚堟垚鏂规硶鍙互鐩存帴閫傜敤浜庡浘鍍忔搷浣滀换鍔°€傛澶?#xff0c;瑙嗚寮曞鐨勫浘鍍忓悎鎴愬拰缂栬緫鍙互搴旂敤浜庤澶氫綆绾ц瑙変换鍔°€備緥濡?#xff0c;鎴戜滑鍙互閫氳繃灏嗙伆搴﹀浘鍍忎綔涓鸿瑙夊紩瀵?#xff0c;灏嗙浉搴旂殑褰╄壊鍥惧儚浣滀负鍩烘湰浜嬪疄鏉ュ疄鐜板浘鍍忕潃鑹?#xff0c;杩樺彲浠ュ簲鐢ㄤ簬鍥惧儚瓒呭垎杈ㄧ巼銆佸浘鍍忓幓闆俱€佸浘鍍忓幓闆ㄧ瓑绛夐鍩?/p>

2.2銆佹枃鏈紩瀵?/h2>

涓庤瑙夊紩瀵?#xff08;濡傝竟缂樺拰瀵硅薄閬僵)鐩告瘮,鏂囨湰寮曞鎻愪緵浜嗕竴绉嶆洿鐏垫椿鐨勬柟寮忔潵琛ㄨ揪瑙嗚姒傚康銆?strong>鏂囨湰鍒板浘鍍忓悎鎴愪换鍔℃棬鍦ㄧ敓鎴愭竻鏅般€佺収鐗囬€肩湡鐨勫浘鍍?#xff0c;涓庣浉搴旂殑鏂囨湰鎸囧崡鍏锋湁楂樺害璇箟鐩稿叧鎬?/strong>銆傝繖椤逛换鍔¢潪甯稿叿鏈夋寫鎴樻€?#xff0c;鍥犱负鏂囨湰鎻忚堪閫氬父鏄ā绯婄殑,骞朵笖鍙兘瀵艰嚧澶ч噺鍏锋湁姝g‘璇箟鐨勫浘鍍忋€?/p>

姝ゅ,鍥惧儚鍜屾枃鏈叿鏈夊紓鏋勭壒寰?#xff0c;杩欎娇寰楀緢闅惧涔犺法涓ょ妯″紡鐨勭簿纭拰鍙潬鏄犲皠銆傚洜姝?#xff0c;瀛︿範鏂囨湰鎻忚堪鐨勭簿纭祵鍏ュ湪鏂囨湰寮曞鐨勫浘鍍忓悎鎴愬拰缂栬緫涓捣鐫€閲嶈浣滅敤銆?/p>

鐢ㄦ枃鏈紩瀵煎悎鎴愬拰缂栬緫鍥惧儚棣栧厛闇€瑕佷粠鏂囨湰琛ㄧず涓涔犳湁鐢ㄧ殑缂栫爜,浼犵粺鏂囨湰缂栫爜鍣ㄦ湁Word2Vec銆丅ag-of-Words,闅忕潃娣卞害绁炵粡缃戠粶鐨勬櫘鍙?#xff0c;Reed绛変汉鎻愬嚭浣跨敤瀛楃绾у嵎绉€掑綊绁炵粡缃戠粶(char-CNN-RNN)闅忕潃鑷劧璇█澶勭悊棰嗗煙涓璁粌妯″瀷鐨勫彂灞?#xff0c;涓€浜涚爺绌禰81]銆乕82]涔熸帰绱簡鍒╃敤澶ц妯¢璁粌璇█妯″瀷(濡侭ERT)杩涜鏂囨湰缂栫爜銆?/p>

鏈€杩?#xff0c;瀵规瘮璇█鍥惧儚棰勮缁?#xff08;CLIP)閫氳繃浠庡ぇ閲忓浘鍍忓拰鏂囨湰瀵逛腑瀛︿範鍥惧儚鍜岀浉搴斿瓧骞曠殑瀵归綈,瀹炵幇浜哠OTA銆傚鍥炬墍绀?#xff0c;CLIP鑱斿悎浼樺寲浜嗘枃鏈紪鐮佸櫒鍜屽浘鍍忕紪鐮佸櫒,浠ユ渶澶у寲姝e涔嬮棿鐨勪綑寮︾浉浼煎害,骞舵渶灏忓寲璐熷鐨勪綑寮︾浉浼兼€?#xff0c;浠庤€屼骇鐢熶俊鎭€ф枃鏈祵鍏?#xff1a;

udp

2.3銆侀煶棰戝紩瀵?/h2>

Harwath绛変汉鎺㈢储浠庤嚜鐒跺浘鍍忓拰鎻忚堪鍥惧儚鐨?strong>鐩稿簲璇煶娉㈠舰涓涔犵缁忕綉缁滃祵鍏ャ€傚涔犳ā鍨嬪厑璁告墽琛岃法璇█璇煶妫€绱€傚0闊充笉浠呭彲浠ヤ笌瑙嗚鍐呭浜や簰,杩樺彲浠ユ崟鑾蜂赴瀵岀殑璇箟淇℃伅銆備緥濡?#xff0c;閫氳繃杞Щ鏉ヨ嚜鍏朵粬棰勮缁冨満鏅拰瀵硅薄璇嗗埆妯″瀷鐨勭煡璇?#xff0c;SoundNet(澹伴煶璇嗗埆鐨勬繁搴︽ā鍨?#xff09;鍙互瀛︿範浠呬娇鐢ㄥ惉瑙夊唴瀹规潵璇嗗埆鍦烘櫙鍜屽璞°€?/p>

瀵瑰簲闊抽缂栫爜鍣ㄥ彲浠ヤ粠缁欏畾瑙嗛鐢熸垚闊抽搴忓垪,鍏朵腑閲囩敤娣卞害鍗风Н缃戠粶浠庤棰戞埅鍥句腑鎻愬彇鐗瑰緛,鐒跺悗浣跨敤LSTM鐢熸垚鐩稿簲杈撳叆瑙嗛鐨勯煶棰戞尝褰€傝緭鍏ラ煶棰戞涔熷彲浠ョ敱涓€绯诲垪鐗瑰緛鏉ヨ〃绀?#xff0c;杩欎簺鐗瑰緛鍙互鏄璋卞浘銆丗BANK鍜宮el棰戠巼鍊掕氨绯绘暟(MFCC),浠ュ強棰勮缁冮煶缃戞ā鍨嬬殑闅愯棌灞傝緭鍑恒€傚浜庨潰閮ㄧ敓鎴?#xff0c;杩樺箍娉涢噰鐢ㄥ姩浣滃崟鍏?#xff08;AU)鏉ュ皢椹卞姩闊抽杞崲涓虹敤浜庨潰閮ㄧ敓鎴愮殑鐩稿共瑙嗚淇″彿銆?/p>

2.4銆佸叾浠栧紩瀵?/h2>

鍦烘櫙鍥?/strong>:鍦烘櫙鍥惧皢鍦烘櫙琛ㄧず涓烘湁鍚戝浘,鍏朵腑鑺傜偣鏄璞?#xff0c;杈圭粰鍑哄璞′箣闂寸殑鍏崇郴銆?strong>鍩轰簬鍦烘櫙鍥剧殑鍥惧儚鐢熸垚鍏佽鎺ㄧ悊鏄惧紡瀵硅薄鍏崇郴骞跺悎鎴愬叿鏈夊鏉傚満鏅叧绯荤殑鐪熷疄鍥惧儚銆傚紩瀵肩殑鍦烘櫙鍥惧彲浠ラ€氳繃鍥惧嵎绉綉缁滅紪鐮?#xff0c;璇ュ浘鍗风Н缃戦娴嬪璞¤竟鐣屾浠ヤ骇鐢熷満鏅竷灞€銆備负浜嗘帹瀵兼瘡涓崟鐙殑涓昏皳瀹捐鍏崇郴,Vo绛変汉寤鸿杩涗竴姝ラ娴嬪璞′箣闂寸殑鍏崇郴鍗曚綅,閫氳繃鍗风НLSTM灏嗗叾杞崲涓鸿瑙夊竷灞€銆?/p>

涓夈€佸妯℃€佸浘鍍忓悎鎴愬拰缂栬緫鏂规硶

澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫(MISE)鐨勬柟娉曞ぇ鑷村垎涓轰簲绫?#xff1a;鍩轰簬GAN鐨勬柟娉?#xff08;3.1)銆佽嚜鍥炲綊鏂规硶(3.2)銆佸熀浜庢墿鏁g殑鏂规硶(3.3)銆佸熀浜庣缁忕綉缁滅殑鏂规硶(3.4)鍜屽叾浠栨柟娉?#xff08;3.5)銆?/p>

3.1銆佸熀浜嶨AN鐨勬柟娉?/h2>

鍩轰簬GAN鐨勬柟娉曞凡琚箍娉涚敤浜庡悇绉峂ISE浠诲姟,閫氳繃寮€鍙戝叿鏈夌壒瀹氬妯℃€佽緭鍏ョ殑鏉′欢GAN鎴栧弽杞棤鏉′欢GAN浠ヤ骇鐢熺洰鏍囨綔鍦ㄤ唬鐮併€備竴鑸皢鍩轰簬GAN鐨勬柟娉曞垎涓轰笁绫?#xff0c;鍖呮嫭鍏锋湁妯℃€佸唴鏉′欢鐨勬柟娉曘€佸叿鏈夎法妯℃€佹潯浠剁殑鏂规硶鍜孏AN鍙嶆紨鏂规硶銆?/p>

3.1.1銆佸叿鏈夋ā鎬佸唴鏉′欢鐨勬柟娉?/h3>

妯℃€佸唴寮曞涓哄浘鍍忓悎鎴愬拰缂栬緫鎻愪緵浜嗘煇浜涜瑙夌嚎绱?#xff08;濡傚満鏅浘銆佸垎鍓插浘銆佽竟缂樺浘绛?#xff09;,浠栦滑鏄緢寮虹殑寮曞鏉′欢,鍙互鏃犻渶閰嶅璁粌鏁版嵁灏卞彲浠ュ疄鐜板悎鎴愬拰缂栬緫銆傚彲浠ュ皢鍏锋湁妯℃€佸唴鏉′欢鐨勬柟娉曞垎涓哄叿鏈夐厤瀵规暟鎹殑鏂规硶鍜屽叿鏈夐潪閰嶅鏁版嵁鐨勩€傚叾涓?#xff1a;

导入导出

  1. 閰嶅鏁版嵁:閰嶅鏁版嵁鎵€鎻愪緵鐨勫紩瀵间即闅忕潃鐩稿簲鐨勭湡瀹炲浘鍍?#xff0c;浠ユ彁渚涗竴瀹氱殑鐩存帴鐩戠潱銆侷sola绛変汉棣栧厛鐮旂┒浜?strong>鏉′欢GAN,灏嗗叾浣滀负涓€涓О涓篜ix2Pix鐨勯€氱敤妗嗘灦,鐢ㄤ簬鍚勭鍥惧儚缈昏瘧浠诲姟(渚嬪,浠庤竟缂樺埌鍥惧儚銆佷粠鐧藉ぉ鍒板鏅氫互鍙婁粠璇箟鍒板浘鍍?#xff09;銆備负浜嗙紦瑙ix2Pix涓珮鍒嗚鲸鐜囧浘鍍忓悎鎴愮殑闄愬埗,Wang绛変汉鎻愬嚭浜哖ix2PixHD,鍏佽鍚堟垚2048脳1024鐨勫浘鍍忋€傜劧鑰?#xff0c;褰撳瓨鍦ㄩ潪甯镐笉鍚岀殑瑙嗗浘鎴栦弗閲嶅彉褰㈡椂,Pix2Pix鍙婂叾鍙樹綋鏃犳硶缂栫爜鍒跺鍜岀湡瀹炲浘鍍忎箣闂寸殑澶嶆潅鍦烘櫙缁撴瀯鍏崇郴銆傚彟涓€鏂归潰,浠ュ墠鐨勬柟娉曠洿鎺ョ敤娣卞害缃戠粶瀵硅瑙夊紩瀵艰繘琛岀紪鐮?#xff0c;浠ヤ究杩涗竴姝ョ敓鎴?#xff0c;杩欐槸娆′紭鐨?#xff0c;鍥犱负寮曞淇℃伅鐨勪竴閮ㄥ垎寰€寰€鍦ㄥ綊涓€鍖栧眰涓涪澶便€係PADE琚璁′负閫氳繃濡備笅鍥炬墍绀虹殑绌洪棿鑷€傚簲褰掍竴鍖?/strong>浠庤€屾湁鏁堝湴娉ㄥ叆寮曞鐗瑰緛銆係EAN寮曞叆浜?strong>璇箟鍖哄煙鑷€傚簲褰掍竴鍖栧眰,浠ュ疄鐜板尯鍩熼鏍兼敞鍏ャ€係haham绛変汉鎻愬嚭浜咥SAPNet鐢ㄤ簬缈昏瘧楂樺垎杈ㄧ巼鍥惧儚鐨勮交閲忕骇浣嗛珮鏁堢殑缃戠粶,Zhang绛変汉鍜孼han绛変汉寮曞叆浜?strong>鍩轰簬绀轰緥鐨勫浘鍍忕炕璇戞鏋?/strong>,璇ユ鏋跺湪绀轰緥鍜屾潯浠惰緭鍏ヤ箣闂村缓绔嬩簡瀵嗛泦鐨勫搴斿叧绯?#xff0c;浠ユ彁渚涘噯纭殑鎸囧銆俍hou绛変汉寤鸿鍒╃敤GRU鍗忓姪鐨勮ˉ涓佸尮閰?/strong>浠ラ珮鏁堝湴寤虹珛楂樺垎杈ㄧ巼鐨勯€氫俊銆俍han绛変汉寮曞叆浜嗕竴绉?strong>鍙屽眰瀵归綈鏂规,浠ラ檷浣庡唴瀛樻垚鏈?#xff0c;鍚屾椂寤虹珛瀵嗛泦鐨勫搴斿叧绯汇€?/li>

2. 闈為厤瀵规暟鎹?/strong>:闈為厤瀵瑰浘鍍忓悎鎴愬埄鐢ㄩ潪閰嶅璁粌鍥惧儚灏嗗浘鍍忎粠涓€涓煙杞崲鍒板彟涓€涓煙銆俍hu绛変汉璁捐浜嗕竴涓惊鐜竴鑷存€ф崯澶?/strong>,浠ラ€氳繃纭繚杈撳叆鍥惧儚鍙互浠庣炕璇戠粨鏋滀腑鎭㈠鏉ヤ繚瀛樺浘鍍忓唴瀹广€傜劧鑰?#xff0c;寰幆闃绘姉鎹熷け瀵逛簬鍥惧儚缈昏瘧鏉ヨ闄愬埗澶ぇ,鍥犱负瀹冨亣璁句袱涓煙涔嬮棿鐨勫弻灏勫叧绯汇€侾ark绛変汉鎻愬嚭閫氳繃鍣0瀵规瘮浼拌鏈€澶у寲姝e鐨勪簰淇℃伅,浠ヤ繚瀛樻湭閰嶅鍥惧儚缈昏瘧涓殑鍐呭銆侫ndonian绛変汉寮曞叆瀵规瘮瀛︿範鏉ユ祴閲忔湭閰嶅鍥惧儚缈昏瘧涓殑鍥惧儚闂寸浉浼煎害銆傜劧鑰?#xff0c;寮傛瀯鍩熼€氬父鍏锋湁鏄犲皠,鍏朵腑涓€涓煙涓殑鍗曚釜鍥惧儚鍦ㄦ槧灏勫悗鍙兘涓庡叾鍦ㄥ彟涓€涓煙鐨勮〃绀轰笉鍏变韩浠讳綍鐗瑰緛銆傚洜姝?#xff0c;TravelGAN鎻愬嚭灏嗗煙鍐呭悜閲忓彉鎹繚瀛樺湪鐢辫繛浣撶綉缁滃涔犵殑娼滃湪绌洪棿涓?/strong>,瀛︿範璺ㄥ紓鏋勫煙鐨勬槧灏勩€?/p>

3.1.2銆佽法妯℃€佹潯浠跺紩瀵肩殑鏂规硶

鍏稿瀷鐨勫熀浜嶨AN鐨勮法妯℃€佸悎鎴愪换鍔℃湁:鏂囨湰鍒板浘鍍忓悎鎴愬拰闊抽椹卞姩鐨勫浘鍍忕紪杈戙€?br />

FreeMarker

  1. 鏂囨湰鐢熸垚鍥惧儚:鏂囨湰鍒板浘鍍忓悎鎴愭棬鍦ㄧ敓鎴愯兘澶熷噯纭弽鏄犳枃鏈弿杩拌涔夌殑鍥惧儚銆俁eed绛変汉鏄涓€涓墿灞曟潯浠禛AN浠ュ疄鐜版枃鏈埌鍥惧儚鍚堟垚鐨勪汉銆傜敱浜嶨ANs鍦ㄥ浘鍍忓悎鎴愭柟闈㈢殑杩涙,璇ヤ换鍔″湪閲囩敤鍫嗗彔鏋舵瀯銆佸惊鐜竴鑷存€у拰娉ㄦ剰鍔涙満鍒?/strong>鏂归潰鍙栧緱浜嗛噸澶ц繘灞曘€傚鍫嗗彔缁撴瀯:StackGAN銆?a href=”https://blog.csdn.net/air__Heaven/article/details/121899032″>StackGAN++銆丠DGAN;寰幆涓€鑷存€?#xff1a;PPGN銆丆ycleGAN銆?a href=”https://blog.csdn.net/air__Heaven/article/details/122795837″>MirrorGAN;娉ㄦ剰鍔涙満鍒?#xff1a;AttnGAN銆丼EGAN銆丆ontrolGAN銆丷iFeGAN;杩欎篃鏄垜姝e湪鐮旂┒鐨勯鍩?#xff0c;鎰熷叴瓒e彲浠ユ繁鍏ユ煡鐪嬩笓鏍?#xff1a;鏂囨湰鐢熸垚鍥惧儚涓撴爮
  2. 闊抽椹卞姩鐨勫浘鍍忕紪杈?#xff1a;鐢变簬鍏跺湪瀹為檯搴旂敤涓殑浠峰€?#xff0c;褰撳墠鐨勯煶棰戦┍鍔ㄥ浘鍍忕紪杈戞柟娉曚晶閲嶄簬璇磋瘽闈㈤儴鐢熸垚闊抽椹卞姩鐨勮璇濋潰閮ㄧ敓鎴愮殑浠诲姟鏃ㄥ湪鍚堟垚璇村嚭缁欏畾闊抽鍓緫鐨勮璇濋潰閮?#xff0c;杩欏湪鏁板瓧闈㈤儴鍔ㄧ敾銆佺數褰卞埗浣溿€佽棰戦厤闊崇瓑鏂归潰鍏锋湁骞挎硾鐨勫簲鐢ㄣ€傞煶棰戦┍鍔ㄧ殑璋堣瘽闈㈤儴鐢熸垚鐨?strong>涓€涓熀鏈寫鎴樻槸濡備綍灏嗛煶棰戝唴瀹瑰噯纭湴杞崲涓鸿瑙変俊鎭?/strong>銆傚埄鐢ㄧ敓鎴愭€у鎶楁ā鍨?#xff0c;Chung绛変汉瀛︿範鍘熷闊抽鍜岃棰戞暟鎹殑鑱斿悎宓屽叆,骞朵娇鐢ㄨВ鐮佸櫒灏嗗叾鎶曞奖鍒板浘鍍忓钩闈?#xff0c;浠ョ敓鎴愯璇濅汉鑴?#xff1b;Zhou绛変汉鎻愬嚭浜嗕竴绉岲A-VS,瀹冨涔犱竴绉嶈В绾犵紶鐨勮鍚〃绀?#xff0c;杩欐湁鍔╀簬鎻愰珮鍚堟垚鐨勮璇濅汉鑴哥殑璐ㄩ噺銆係ong绛変汉浠嬬粛浜嗕竴绉嶆潯浠禦NN缃戠粶,鐢ㄤ簬瀵规姉鎬у湴鐢熸垚瀵硅瘽浜鸿劯銆侰hen绛変汉璁捐浜嗕竴绉嶅垎灞傜粨鏋?#xff0c;灏嗛煶棰戝壀杈戞槧灏勫埌闈㈤儴鍦版爣涓?#xff0c;杩涗竴姝ュ埄鐢ㄨ繖浜涘湴鏍囨潵鐢熸垚浼氳璇濈殑闈㈤儴銆俍hou绛変汉浠嬬粛浜哅akeItTalk,瀹冧粠璇煶鍐呭涓娴嬭璇濅汉鎰熺煡鐨勯潰閮ㄦ爣蹇?#xff0c;浠ユ洿濂藉湴淇濈暀璇磋瘽浜虹殑鐗瑰緛銆?#xff0c;Yi绛変汉鎻愬嚭灏嗛煶棰戝唴瀹规槧灏勫埌3DMM鍙傛暟,鐢ㄤ簬寮曞濮垮娍鍙帶鐨勭敓鎴愯皥璇濋潰閮?#xff1b;Zhou绛変汉鎻愬嚭浜嗕竴绉峆C-AVS,瀹冮€氳繃瀛︿範濮垮娍銆佽韩浠藉拰璇煶鍐呭鐨勫垎绂荤壒寰佺┖闂存潵瀹炵幇濮垮娍鍙帶鐨勮皥璇濋潰閮ㄧ敓鎴愩€?/li>

3.1.3銆丟AN鐨勯€嗘槧灏?/h3>

鍒╃敤棰勫厛璁粌鐨凣AN妯″瀷,涓€绯诲垪鐮旂┒鎺㈢储灏嗙粰瀹氬浘鍍忓弽杞洖GAN鐨勬綔鍦ㄧ┖闂?#xff0c;杩欒绉颁负GAN閫嗘槧灏勩€?strong>GAN閫嗘槧灏勫皢鍥惧儚鏄犲皠鍥炴綔鍦ㄧ┖闂?#xff0c;閫氳繃灏嗘綔绌洪棿棣堝叆棰勮缁冪殑GAN浠ラ€氳繃浼樺寲閲嶅缓鍥惧儚鏉ュ疄鐜?/strong>銆傚湪澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫鏂归潰,鍩轰簬GAN閫嗘槧灏勬柟娉曠殑鍏抽敭鍦ㄤ簬濡備綍鏍规嵁鐩稿簲鐨勬寚鍗楃紪杈戞垨鐢熸垚娼滃湪浠g爜,涓斿熀浜庡弽婕旂殑鏂规硶寰€寰€瀵规ā鎬佷笉澶晱鎰?#xff0c;鍥犱负鍙嶆紨鏄湪闈炵粨鏋勫寲涓€缁存綔鍦ㄧ┖闂翠笂杩涜鐨勩€?br />

快捷键

娼滃湪绌洪棿閲岀殑璺ㄦā鎬佸尮閰?/strong>:澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫鍙互閫氳繃鍦ㄥ叕鍏卞祵鍏ョ┖闂翠腑鍖归厤鍥惧儚宓屽叆鍜岃法妯℃€佽緭鍏?#xff08;渚嬪,璇箟鍥俱€佹枃鏈?#xff09;鏉ュ疄鐜?#xff0c;鍏蜂綋鑰岃█,璺ㄦā鎬佺紪鐮佸櫒琚缁冧互瀛︿範鍏锋湁瑙嗚璇█鐩镐技鎬ф崯澶卞拰鎴愬鎺掑簭鎹熷け鐨勫祵鍏?/strong>銆備负浜嗗湪缂栬緫鍚庝繚鎸佽韩浠?#xff0c;鍙互鍦ㄧ洰鏍囦腑浣跨敤瀹炰緥绾т紭鍖栨ā鍧?#xff0c;璇ユā鍧楀厑璁告牴鎹枃鏈弿杩颁慨鏀圭洰鏍囧睘鎬с€傜敱浜庡湪StyleGAN娼滅┖闂翠腑鎵ц浼樺寲,璇ユ鏋跺浐鏈夊湴鍏佽浠庣粰瀹氱殑澶氭ā鎬佹潯浠剁敓鎴愬浘鍍忋€傛潯浠跺浘鍍忔搷浣滀篃鍙互閫氳繃鍏变韩娼滃湪绌洪棿涓殑鏍峰紡娣峰悎鏉ユ墽琛屻€?/p>

娼滃湪绌洪棿涓殑鍥惧儚缂栫爜浼樺寲:涓庡皢鏂囨湰鏄犲皠鍒版綔鍦ㄧ┖闂翠笉鍚?#xff0c;涓€涓祦琛岀殑鐮旂┒璺嚎鏃ㄥ湪鐩存帴浼樺寲鍘熷鍥惧儚鐨勬綔鍦ㄤ唬鐮?#xff0c;骞朵互娴嬮噺璺ㄦā鎬佷竴鑷存€х殑鏌愪簺鎹熷け涓烘寚瀵笺€傞儴鍒嗙爺绌朵娇鐢?strong>瀵规瘮璇█鍥惧儚棰勮缁?/strong>(CLIP)鏉ユ寚瀵兼枃鏈埌鍥惧儚鍚堟垚鐨勯€嗘槧灏勮繃绋?#xff0c;鑰屼笉鏄娇鐢ㄧ壒瀹氱殑灞炴€ч娴嬪櫒銆傞拡瀵规枃鏈紩瀵肩殑鍥惧儚淇,Bau绛変汉瀹氫箟浜?strong>鍩轰簬鍓緫鐨勮涔変竴鑷存€ф崯澶?/strong>,璇ュ壀杈戜紭鍖栦簡淇鍖哄煙鍐呯殑娼滃湪浠g爜,浠ュ疄鐜颁笌缁欏畾鏂囨湰鐨勮涔変竴鑷淬€係tyleClip鍜孲tyleMC浣跨敤棰勮缁冪殑鍓緫浣滀负涓㈠け鐩戠潱,浠ュ皢鎿嶇旱缁撴灉涓庢枃鏈潯浠跺尮閰?#xff0c;濡傚浘鎵€绀恒€傞€氳繃寮曞叆鐢ㄤ簬鍙嶄簨瀹炲浘鍍忔搷绾电殑鍩轰簬鍓緫鐨勫姣斾涪澶便€佺敤浜庣ǔ鍋ユ€ф爣鍑嗗壀杈戝垎鏁扮殑AugCLIP鍒嗘暟,杩囧弬鏁板寲绛栫暐,浠ュ湪娼滃湪绌洪棿涓鑸紭鍖栥€?br />

srio

3.2銆佽嚜鍥炲綊鏂规硶

Transformer妯″瀷鍒╃敤鍏跺己澶х殑娉ㄦ剰鍔涙満鍒跺凡鎴愪负搴忓垪鐩稿叧寤烘ā鐨勮寖渚?#xff0c;鍙桮PT妯″瀷鍦ㄨ嚜鐒惰瑷€寤烘ā涓殑鎴愬姛鍚彂,鍥惧儚GPT(iGPT)閫氳繃灏嗗睍骞冲浘鍍忓簭鍒楄涓虹鏁f爣璁?#xff0c;閲囩敤Transformer杩涜鑷洖褰掑浘鍍忕敓鎴愩€傜敓鎴愬浘鍍忕殑鍚堢悊鎬ц〃鏄?#xff0c;Transformer妯″瀷鑳藉妯℃嫙鍍忕礌鍜岄珮绾у睘鎬?#xff08;绾圭悊銆佽涔夊拰姣斾緥)涔嬮棿鐨勭┖闂村叧绯汇€?br />
鐢变簬Transformer妯″瀷鍥烘湁鍦版敮鎸佸妯℃€佽緭鍏?#xff0c;鎵€浠ュ緢澶氱爺绌朵篃鍦ㄦ帰绱㈠熀浜庡彉鍘嬪櫒鐨勮嚜鍥炲綊妯″瀷鐨勫妯℃€佸浘鍍忓悎鎴愩€傚熀浜嶵ransformer鐨勫浘鍍忓悎鎴愮殑娴佺▼鍖呮嫭瀹炵幇绂绘暎琛ㄧず鍜屾暟鎹淮搴﹀帇缂╃殑鐭㈤噺閲忓寲杩囩▼,浠ュ強浠ュ厜鏍呮壂鎻忛『搴忓缓绔嬬鏁f爣璁颁箣闂寸浉鍏虫€х殑鑷洖褰掑缓妯¤繃绋嬨€?/p>

3.2.1銆佸浘鍍忕煝閲忛噺鍖?/h3>

浣跨敤Transformer鐩存帴灏?strong>鎵€鏈夊浘鍍忓儚绱犱綔涓哄簭鍒楄繘琛岃嚜鍥炲綊寤烘ā鍦ㄥ唴瀛樻秷鑰楁柟闈㈡槸鏋佷负楂樼殑,鍥犱负鑷敞鎰忓姏鏈哄埗鍦═ransformer閲屼骇鐢熶簡浜屾鍐呭瓨鎴愭湰銆傚洜姝?#xff0c;鍥惧儚鐨勫帇缂╁拰绂绘暎琛ㄧず瀵逛簬鍩轰簬鍙樻崲鍣ㄧ殑鍥惧儚鍚堟垚鍜岀紪杈戣嚦鍏抽噸瑕併€?/p>

Chen绛変汉閲囩敤璋冭壊鏉垮皢缁存暟闄嶄綆鍒?12,鍚屾椂淇濈暀鍘熷鍥惧儚鐨勪富瑕佺粨鏋?#xff0c;鐒惰€?#xff0c;k鍧囧€艰仛绫讳粎鍑忓皯浜嗙爜鏈淮鏁扮殑澶у皬,浣嗗簭鍒楅暱搴︿粛鐒朵笉鍙樸€備负姝?#xff0c;鐭㈤噺閲忓寲VAE(VQ-VAE)琚箍娉涚敤浜庡涔犲浘鍍忕殑绂绘暎鍜屽帇缂╄〃绀恒€?strong>涓轰簡瀛︿範浼樼鐨勭鏁e拰鍘嬬缉鍥惧儚琛ㄧず,鍦ㄦ崯澶辫璁°€佺綉缁滅粨鏋勫拰姝e垯鍖栨柟闈?#xff0c;宸茬粡鍋氬嚭浜嗕竴绯诲垪鍔姏鏉ユ敼杩沄Q-VAE:

方法重载与方法重写的区别

  1. 鎹熷け鍑芥暟璁捐:涓轰簡瀹炵幇閲嶅缓鍥惧儚鐨勮壇濂芥劅鐭ヨ川閲?#xff0c;Esser绛変汉鎻愬嚭浜嗕竴绉峍QGAN,璇QGAN缁撳悎浜?strong>鍩轰簬琛ヤ竵鐨勯壌鍒櫒鐨勫鎶楁€ф崯澶卞拰鐢ㄤ簬鍥惧儚閲嶅缓鐨勬劅鐭ユ崯澶?/strong>,Dong绛変汉鍒╃敤鑷洃鐫g綉缁滅敤浜庡涔犳繁搴﹁瑙夌壒寰?/strong>,浠ュ湪dVAE璁粌鏈熼棿澧炲己鎰熺煡鐩镐技鎬с€傜敱浜庨澶栫殑瀵规姉鎬ф崯澶卞拰鎰熺煡鎹熷け,涓庡浘鍍忛噸寤轰腑鐨勫師濮嬪儚绱犳崯澶辩浉姣?#xff0c;鍥惧儚璐ㄩ噺鏄庢樉鎻愰珮銆侴afni绛変汉鍦ㄩ璁粌浜鸿劯宓屽叆缃戠粶鐨勬縺娲昏繃绋嬩腑閲囩敤浜?strong>鐭㈤噺閲忓寲鐗瑰緛鍖归厤鎹熷け
  2. 缃戠粶鏋舵瀯:Yu绛変汉鎻愬嚭浜哣iT VQGAN,鐢ㄨ瑙夊彉鎹㈠櫒(ViT)浠f浛CNN缂栫爜鍣ㄥ拰瑙g爜鍣?/strong>銆傜粰瀹氳冻澶熺殑鏁版嵁(鍏朵腑鏈爣璁扮殑鍥惧儚鏁版嵁闈炲父涓板瘜),ViT VQGAN琚瘉鏄庝笉鍙楀嵎绉柦鍔犵殑褰掔撼鍏堥獙鐨勭害鏉?#xff0c;骞朵笖鑳藉浠ユ洿楂樼殑閲嶅缓璐ㄩ噺浜х敓鏇村ソ鐨勮绠楁晥鐜?/strong>銆俈iT VQGAN杩樻彁鍑轰簡涓€绉?strong>鍥犲紡鍒嗚В浠g爜鏋舵瀯,璇ユ灦鏋勫紩鍏ヤ簡浠庣紪鐮佸櫒杈撳嚭鍒颁綆缁存綔鍦ㄥ彉閲忕┖闂寸殑绾挎€ф姇褰?#xff0c;鐢ㄤ簬浠g爜绱㈠紩鏌ユ壘,骞跺ぇ澶ф彁楂樹簡鐮佹湰浣跨敤鐜囥€傛澶?#xff0c;NUWA-LIP鎺㈢储浜嗕竴绉?strong>澶氳瑙掔紪鐮?/strong>,閫氳繃鍚屾椂鍖呭惈浣庣骇鍍忕礌鍜岄珮绾ф爣璁版潵澧炲己瑙嗚淇℃伅;DiVAE閲囩敤鍩轰簬鎵╂暎鐨勮В鐮佸櫒瀛︿範鍏锋湁鍗撹秺鍥惧儚閲嶅缓鎬ц兘鐨勭鏁e浘鍍忚〃绀?#xff1b;
  3. 姝e垯鍖?#xff1a;Shin绛変汉楠岃瘉浜唙anilla VQ-VAE鍦ㄩ噺鍖栬繃绋嬩腑涓嶆弧瓒冲钩绉荤瓑鍙?#xff0c;瀵艰嚧鏂囨湰鍒板浘鍍忕敓鎴愭€ц兘涓嬮檷銆傚洜姝?#xff0c;鎻愬嚭浜嗕竴绉嶇畝鍗曚絾鏈夋晥鐨凾E-VQGAN,閫氳繃姝e垯鍖栫爜鏈祵鍏ヤ腑鐨勬浜ゆ€ф潵瀹炵幇骞崇Щ绛夊彉銆備负浜?strong>鍦ㄥ涓煙涓疄鐜版潯浠跺浘鍍忕敓鎴愮殑鑱斿悎閲忓寲,Zhan绛変汉璁捐浜嗗叿鏈夊彉鍒嗘鍒欏寲瀛愮殑闆嗘垚閲忓寲VAE(IQ-VAE),浠ユ鍒欏寲璺ㄥ煙绌洪棿涓殑鐗瑰緛閲忓寲

3.2.2銆佽嚜鍥炲綊妯″瀷

Mask R-CNN

  1. 婊戝姩绐楀彛閲囨牱:涓轰簡鍔犻€熻嚜鍥炲綊鍥惧儚鐢熸垚,Esser绛変汉閲囩敤婊戝姩绐楀彛绛栫暐浠庡浘鎵€绀虹殑璁粌鍙樺帇鍣ㄦā鍨嬩腑杩涜閲囨牱銆?strong>婊戝姩绐楀彛绛栫暐浠呭埄鐢ㄥ眬閮ㄧ獥鍙e唴鐨勯娴?#xff0c;鑰屼笉鏄埄鐢ㄦ墍鏈夊厛鍓嶉娴嬫潵浼拌褰撳墠缁撴灉,杩欐樉钁楀噺灏戜簡鎺ㄦ柇鏃堕棿銆傚彧瑕佺┖闂存潯浠朵俊鎭彲鐢ㄦ垨鏁版嵁闆嗙粺璁℃暟鎹繎浼肩┖闂翠笉鍙?#xff0c;婊戝姩绐楀彛涓殑灞€閮ㄤ笂涓嬫枃灏辫冻浠ュ鍥惧儚搴忓垪杩涜寤烘ā;
  2. 鍙屽悜涓婁笅鏂?/strong>:浠ュ墠鐨勬柟娉曢€氳繃鍙叧娉ㄤ笂涓€浠g粨鏋?#xff0c;灏嗗浘鍍忎笂涓嬫枃鍚堝苟鍒板厜鏍呮壂鎻忛『搴忎腑,瀹冪殑绛栫暐鏄崟鍚戠殑,蹇界暐浜嗗ぇ閲忎笂涓嬫枃淇℃伅,瀛樺湪椤哄簭鍋忓樊銆傚畠杩樺拷鐣ヤ簡涓嶅悓灏哄害涓嬬殑璁稿涓婁笅鏂囦俊鎭?#xff0c;鍥犱负瀹冨彧鍦ㄥ崟涓昂搴︿笂澶勭悊鍥惧儚銆傚熀浜庝笂杩拌瀵?#xff0c;ImageBART鍦ㄧ粺涓€妗嗘灦涓彁鍑轰簡涓€绉嶄粠绮楀埌绮剧殑鏂规硶,瑙e喅浜嗚嚜鍥炲綊寤烘ā鐨勫崟鍚戝亸宸拰鐩稿簲鐨勬洕鍏夊亸宸€傚浜庡浘鍍忕殑鍘嬬缉涓婁笅鏂囦俊鎭?#xff0c;搴旂敤鎵╂暎杩囩▼浠ヨ繛缁秷闄や俊鎭?#xff0c;浜х敓琛ㄧず鐨勫眰娆$粨鏋?#xff0c;鍏堕€氳繃澶氶」寮忔墿鏁h繃绋嬭繘涓€姝ュ帇缂?#xff0c;閫氳繃瀵归┈灏斿彲澶浆绉昏嚜鍥炲綊寤烘ā,骞舵敞鎰忓埌鍓嶉潰鐨勫垎灞傜姸鎬?#xff0c;鍙互涓烘瘡涓崟鐙殑鑷洖褰掓楠ゅ埄鐢ㄥ叧閿殑鍏ㄥ眬涓婁笅鏂囥€備綔涓烘浛浠f柟妗?#xff0c;杩樺箍娉涙帰绱簡鍙屽悜鍙樻崲鍣?#xff0c;浠ョ粨鍚堝弻鍚戜笂涓嬫枃,骞朵即鏈夋帺钄借瑙変护鐗屽缓妯?#xff08;MVTM)鎴栨帺钄借瑷€寤烘ā(MLM)鏈哄埗;
  3. 鏇村ソ鐨勮嚜娉ㄦ剰鍔涙満鍒?/strong>:涓轰簡浠ョ粺涓€鐨勬柟寮忓鐞嗕笉鍚屼换鍔′腑鐨勮瑷€銆佸浘鍍忓拰瑙嗛,NUWA鎻愬嚭浜嗕竴绉嶅叿鏈夌粺涓€鐨?strong>3D閭昏繎鑷垜娉ㄦ剰鍔?/strong>(3DNA)鐨?D杞崲鍣ㄦ鏋?#xff0c;璇ユ鏋朵笉浠呴檷浣庝簡鍏ㄦ敞鎰忓姏鐨勫鏉傛€?#xff0c;鑰屼笖鏄剧ず浜嗕紭瓒婄殑鎬ц兘;
  4. 妯″瀷鏋舵瀯:涓轰簡鎺㈢储鑷洖褰掓枃鏈浘鍍忓悎鎴愮殑灞€闄愭€?#xff0c;Parti灏員ransformer鐨勫弬鏁板ぇ灏忕缉鏀捐嚦20B,骞惰瀵熷埌鍥惧儚璐ㄩ噺鍜屾枃鏈浘鍍忓榻愭柟闈㈢殑涓€鑷磋川閲忔敼杩涖€侶uang绛変汉涓嶆槸浠庢枃鏈埌鍥惧儚鐨勫崟鍚戝缓妯?#xff0c;鑰屾槸绗竴涓彁鍑轰簡涓€绉嶅叿鏈塗ransformer鐨勫弻鍚戝浘鍍忓拰鏂囨湰妗嗘灦,璇ユ鏋跺彲鐢熸垚澶氫釜涓嶅悓鐨勫瓧骞曞拰鍥惧儚銆?/li>

3.3銆佹墿鏁fā鍨?/h2>

鍘诲櫔鎵╂暎姒傜巼妯″瀷(DDPM)鏄竴绉嶆綔鍦ㄥ彉閲忔ā鍨?#xff0c;鍖呮嫭姝e悜鎵╂暎杩囩▼鍜屽弽鍚戞墿鏁h繃绋嬨€?/p>

3.3.1銆佹潯浠舵墿鏁fā鍨?/h3>

涓轰簡鍚姩澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫(MISE)浠诲姟,鍙互閫氳繃灏嗘潯浠朵俊鎭笌鍣0鍥惧儚鐩存帴杩炴帴浣滀负鍘诲櫔缃戠粶鐨勮緭鍏?#xff0c;鑷劧鍦板鍑烘潯浠舵墿鏁fā鍨嬨€傛渶杩?#xff0c;涓€绯诲垪璁捐鏋佸ぇ鍦版帹鍔ㄤ簡鏉′欢鎵╂暎妯″瀷鐨勬€ц兘銆?/p>

  1. 鏃犲垎绫诲櫒鍒跺:鍒嗙被鍣ㄥ紩瀵肩殑缂虹偣鍦ㄤ簬闇€瑕侀澶栫殑鍒嗙被鍣ㄦā鍨?#xff0c;鑰岃繖浼氬鑷村鏉傜殑璁粌绠¢亾銆傛渶杩?#xff0c;Ho绛変汉閫氳繃浣跨敤鏃犲垎绫诲櫒寮曞,鍦ㄦ病鏈夊崟鐙缁冨垎绫诲櫒鐨勬儏鍐典笅鍙栧緱浜嗕护浜轰俊鏈嶇殑缁撴灉,鏃犲垎绫诲櫒寮曞鏄竴绉嶅紩瀵煎舰寮?#xff0c;鍦ㄦ湁鍜屾棤鏍囩鐨勬墿鏁fā鍨嬮娴嬩箣闂磋繘琛屾彃鍊?/strong>銆傚湪杩欎竴鐮旂┒鑼冨洿鍐?#xff0c;GLIDE姣旇緝浜嗘枃鏈紩瀵煎浘鍍忓悎鎴愭墿鏁fā鍨嬩腑鐨勫壀杈戝紩瀵煎拰鏃犲垎绫诲櫒寮曞,骞跺緱鍑虹粨璁?#xff0c;鏃犲垎绫诲櫒寮曞浜х敓鏇村ソ鐨勬€ц兘,35浜垮弬鏁扮殑鎵╂暎妯″瀷鍦ㄤ汉绫昏瘎浼版柟闈紭浜嶥ALL-E銆傛澶?#xff0c;Tang绛変汉鎺㈣浜嗙鏁e幓鍣墿鏁fā鍨嬬殑鏃犲垎绫诲櫒寮曞閲囨牱,寮曞叆浜嗘棤鍒嗙被鍣ㄥ紩瀵肩殑鏈夋晥瀹炵幇;
  2. 鍚堝苟妯″瀷:DALL-E 2棣栧厛浠庢枃鏈爣棰樼敓鎴怌LIP鍥惧儚宓屽叆,鐒跺悗浣跨敤鎵╂暎瑙g爜鍣ㄧ敓鎴愪互鍥惧儚宓屽叆涓烘潯浠剁殑鍥惧儚,鑰屼笉鏄洿鎺ラ檺鍒舵潯浠跺祵鍏?#xff0c;Wang绛変汉鎻愬嚭閫氳繃绌洪棿鑷€傚簲褰掍竴鍖栧皢璇箟鏄犲皠鎻愪緵缁欒В鐮佸櫒,杩欐彁楂樹簡鐢熸垚鍥惧儚鐨勮川閲忓拰璇箟涓€鑷存€?#xff1b;Zhu閫氳繃瀵规瘮瀛︿範鏈€澶у寲鏉′欢鍜岀敓鎴愮殑杈撳嚭涔嬮棿鐨勪簰淇℃伅,浠庤€岀‘淇濇潯浠跺拰鐢熸垚杈撳嚭涔嬮棿鐨勫搴斿叧绯?#xff1b;
  3. 妯″瀷鏋舵瀯:鍩轰簬Transformer鍦ㄧ悊瑙f枃鏈柟闈㈢殑鑳藉姏,Imagen鍦ㄥ熀浜庢墿鏁g殑鏂规硶涓疄鐜颁簡SOTA鏂囨湰鍒板浘鍍忕殑鐢熸垚鎬ц兘,骞跺彂鐜板湪浠呮枃鏈鏂欏簱涓婇璁粌鐨勯€氱敤澶у瀷璇█妯″瀷(渚嬪T5)鍦ㄧ紪鐮佸浘鍍忓悎鎴愭枃鏈柟闈㈠嚭浜烘剰鏂欏湴鏈夋晥銆備负浜嗙悊瑙e満鏅腑鐨勫悎鎴愭蹇?#xff0c;Liu绛変汉鎻愬嚭浜嗕竴绉?strong>鍩轰簬鎵╂暎鐨勫浘鍍忓悎鎴愮殑鍚堟垚鏋舵瀯,璇ユ灦鏋?strong>閫氳繃鍚堟垚涓€缁勬墿鏁fā鍨嬫潵鐢熸垚鍥惧儚銆傝瀵熷埌鎵╂暎妯″瀷鐨勬垚鍔熷緢澶х▼搴︿笂鏄敱浜庤缁冩垚鏈殑澶у箙澧炲姞,Blattmann绛変汉鎻愬嚭浜嗕竴绉?strong>鍩轰簬妫€绱㈢殑鏂规硶鏉ヨˉ鍏呮墿鏁fā鍨?/strong>,璇ユ柟娉曚骇鐢熶簡杈冧綆鐨勮绠楁垚鏈?#xff1b;
  4. 绂绘暎鎵╂暎:涓轰簡浣挎墿鏁fā鍨嬭兘澶熷湪鏈夐檺鐨勮绠楄祫婧愪笂杩涜璁粌,鍚屾椂淇濇寔鍏惰川閲忓拰鐏垫椿鎬?#xff0c;涓€浜涘伐浣滄帰绱㈠湪濡傚浘鎵€绀虹殑VQ-VAE鐨勬綔鍦ㄧ┖闂翠腑杩涜鎵╂暎杩囩▼銆傞€氳繃鍦ㄥ叿鏈夎В鐮佸櫒鍚告敹鐨勯噺鍖栧眰鐨刅QGAN鍙樹綋鐨勬綔鍦ㄧ┖闂翠笂璁粌鎵╂暎妯″瀷,娼滃湪鎵╂暎妯″瀷(LDM)鏄涓€涓湪澶嶆潅搴﹂檷浣庡拰缁嗚妭淇濇寔涔嬮棿瀹炵幇鎺ヨ繎鏈€浣崇偣鐨勫伐浣?#xff0c;鏋佸ぇ鍦版彁楂樹簡瑙嗚淇濈湡搴︺€傜被浼煎湴,Gu绛変汉閫氳繃浣跨敤鍘诲櫔鎵╂暎姒傜巼妯″瀷(DDPM)鐨勬潯浠跺彉閲忓涔犲弬鏁版ā鍨?#xff0c;鎻愬嚭浜嗙敤浜庢枃鏈埌鍥惧儚鐢熸垚鐨勭煝閲忛噺鍖栨墿鏁?#xff08;VQ鎵╂暎)妯″瀷銆俆ang绛変汉閫氳繃寮曞叆楂樿川閲忔帹鐞嗙瓥鐣ユ潵缂撹В鑱斿悎鍒嗗竷闂,杩涗竴姝ユ敼杩涗簡VQ鎵╂暎銆傚湪VQ鎵╂暎涔嬪悗,寮曞叆浜員ext2Human,閫氳繃浣跨敤鍩轰簬鎵╂暎鐨凾ransformer瀵瑰垎灞傜鏁f綔鍦ㄧ┖闂村缓妯?#xff0c;瀹炵幇楂樿川閲忕殑鏂囨湰椹卞姩鐨勪汉绫荤敓鎴愩€?br />

3.3.2銆侀璁粌鎵╂暎妯″瀷

鏉′欢鎵╂暎妯″瀷闇€瑕佷粠澶村紑濮嬮噸鏂拌缁?#xff0c;杩欏闄㈣緝楂樼殑璁$畻鎴愭湰,鍥犳閮ㄥ垎鐮旂┒濡傞€嗘槧灏凣AN涓€鏍锋帰绱簡鍙嶈浆棰勮缁冩墿鏁fā鍨嬩互瀹炵幇鏉′欢鐢熸垚銆侱hariwal绛変汉浣跨敤鍒嗙被鍣ㄥ紩瀵煎寮轰簡鎵╂暎妯″瀷,璇ュ垎绫诲櫒寮曞鍏佽浠庡垎绫诲櫒鏍囩鐢熸垚鏉′欢銆侹im绛変汉鎻愬嚭浜嗕竴绉?strong>鐢ㄤ簬鏂囨湰椹卞姩鐨勫浘鍍忔搷绾电殑鎵╂暎妯″瀷DiffusionCLIP,璇ユā鍨嬮噰鐢ㄩ璁粌鎵╂暎妯″瀷,浣跨敤CLIP鎹熷け鏉ュ皢缂栬緫杞悜缁欏畾鐨勬枃鏈彁绀?#xff0c;

鐢变簬CLIP鏄湪骞插噣鍥惧儚涓婅缁冪殑,鍥犳鍦ㄥ幓鍣墿鏁h繃绋嬩腑,鏈熸湜鏈変竴绉嶄粠甯﹀櫔娼滃湪绌洪棿浼拌骞插噣鍥惧儚鐨勬柟娉曘€傚叿浣撹€岃█,Liu绛変汉寮曞叆浜?strong>鏃犳枃鏈敞閲婄殑鍓緫鑷洃鐫e井璋?/strong>,浠ュ己鍒朵粠骞插噣鍜屾湁鍣浘鍍忎腑鎻愬彇鐨勭壒寰佷箣闂村榻愩€傞樋澶媺鍝堢背绛変汉鎻愬嚭浜嗕竴绉嶅湪鍘诲櫔鎵╂暎杩囩▼涓粠娼滃湪鍣0涓及璁″共鍑€鍥惧儚鐨勬柟娉曘€傜劧鍚?#xff0c;鍩轰簬CLIP鐨勬崯澶卞彲浠ュ畾涔変负鏂囨湰鎻愮ず鐨勫壀杈戝祵鍏ュ拰浼拌鐨勫共鍑€鍥惧儚鐨勫祵鍏ヤ箣闂寸殑浣欏鸡璺濈銆傛澶?#xff0c;涓轰簡瀹炵幇鍥惧儚缂栬緫鐨勫眬閮ㄦ寚瀵?#xff0c;Avrahami绛変汉浠呰€冭檻杈撳叆鎺╂ā涓嬬殑CLIP姊害銆傚湪CLIP寮曞鎵╂暎涓彁鍑轰簡涓€绉嶇被浼肩殑浼拌鏂规硶,鍏朵腑鍣0娼滃湪鍜岄浼扮殑骞插噣鍥惧儚宓屽叆鐨勭嚎鎬х粍鍚堢敤浜庝负鎵╂暎鎻愪緵鍏ㄥ眬鎸囧銆?/p>

3.4銆佸熀浜嶯eRF鐨勬柟娉?/h2>

绁炵粡杈愬皠鍦?/strong>(NeRF)閫氳繃浣跨敤绁炵粡缃戠粶瀹氫箟闅愬紡鍦烘櫙琛ㄧず,瀹炵幇浜嗕护浜哄嵃璞℃繁鍒荤殑鏂拌鍥惧悎鎴愭€ц兘銆傜壒鍒槸,NeRF閲囩敤鍏ㄨ繛鎺ョ缁忕綉缁?/strong>,灏嗙┖闂翠綅缃?#xff08;x,y,z)鍜岀浉搴旂殑瑙傚療鏂瑰悜(胃,蠁))浣滀负杈撳叆,灏嗕綋绉瘑搴﹀拰鐩稿簲鐨勫彂灏勮緪灏勪綔涓鸿緭鍑恒€備负浜嗕粠闅愬紡3D琛ㄧず涓覆鏌?D鍥惧儚,浣跨敤鏁板€肩Н鍒嗗櫒鎵ц鍙井鍒嗕綋绉覆鏌?#xff0c;浠ヨ繎浼奸毦浠ュ鐞嗙殑浣撶Н鎶曞奖绉垎銆傜劧鍚?#xff0c;鍙互閫氳繃娓叉煋鍥惧儚鍜岀浉搴旂殑鍦伴潰鐪熷疄鍥惧儚涔嬮棿鐨勫厜搴︽崯澶辨潵浼樺寲NeRF銆傜敱鐢ㄤ簬3D鍦烘櫙琛ㄧず鐨凬eRF鎻愪緵鍔ㄥ姏,鍙互閫氳繃浣跨敤澶氳鍥剧洃鎺т紭鍖栨瘡涓満鏅殑NeRF銆佸湪鍗曠洰鍥惧儚涓婅缁冪敓鎴怤eRF鎴栧弽杞璁粌鐨勭敓鎴怤eRF鏉ュ疄鐜?D鎰熺煡MISE銆?/p>

3.4.1銆?Per-scene浼樺寲NeRF

涓庢渶鍒濈殑NeRF涓€鑷?#xff0c;璁稿浣滃搧鍏虫敞鍗曚釜鍦烘櫙鐨勯殣寮忚〃绀恒€傚湪鏂囨湰椹卞姩鐨?D鎰熺煡鍥惧儚鍚堟垚涓?#xff0c;Jain绛変汉寮曞叆浜咲ream Fields,鍏跺埄鐢ㄩ璁粌鐨勫浘鍍忔枃鏈ā鍨嬩紭鍖栫缁忚緪灏勫満,NeRF琚紭鍖栦互鏍规嵁棰勮缁冪殑CLIP妯″瀷寮曞叆浜嗙畝鍗曠殑鍑犱綍鍏堥獙,鍖呮嫭绋€鐤忚瀵奸€忓皠姝e垯鍖栥€佸満鏅竟鐣屽拰鏂扮殑MLP鏋舵瀯,鍛堢幇鍏锋湁鐩爣鏂囨湰鎻忚堪鐨勯珮寰楀垎鐨勫瑙嗗浘鍥惧儚銆?br />
浣滀负闊抽椹卞姩鍥惧儚缂栬緫鐨勬墿灞?#xff0c;AD NeRF閫氳繃浣跨敤涓€涓洰鏍囦汉鐨勯煶杞ㄥ湪瑙嗛搴忓垪涓婅缁冪缁忚緪灏勫満,瀹炵幇楂樹繚鐪熺殑璇磋瘽浜哄悎鎴?/strong>銆備笌鍏堝墠鍩轰簬涓棿琛ㄧず妗ユ帴闊抽杈撳叆鍜岃棰戣緭鍑虹殑鏂规硶涓嶅悓,AD-NeRF灏嗛煶棰戠壒寰佺洿鎺ラ閫佸埌闅愬紡鍑芥暟涓?#xff0c;浠ヤ骇鐢熷姩鎬佺缁忚緪灏勫満,杩涗竴姝ュ埄鐢ㄨ杈愬皠鍦洪€氳繃浣撶粯鍒跺悎鎴愪即闅忛煶棰戠殑楂樹繚鐪熻璇濋潰閮ㄨ棰戙€?/p>

NeuS鎺㈢储缁撳悎SDF鍜孨eRF鐨勪紭鐐?#xff0c;浠ュ疄鐜伴珮璐ㄩ噺鐨勮〃闈㈤噸寤恒€侶ong绛変汉灏哊eu浣滀负鍖栬韩鐨勫熀鏈〃绀?#xff0c;AvatarCLIP瀹炵幇鐢ㄤ簬3D鍖栬韩鐢熸垚鍜屽姩鐢荤殑闆堕暅澶存枃鏈┍鍔ㄦ鏋躲€傚埄鐢ㄩ璁粌鐗囨鏉ョ洃鐫g缁忎汉浣撶敓鎴?#xff0c;鍖呮嫭3D鍑犱綍銆佺汗鐞嗗拰鍔ㄧ敾銆?/p>

3.4.2銆佺敓鎴愭€eRF

涓嶱er-scene浼樺寲NeRF涓嶅悓鐨勬槸,鐢熸垚鎬eRF闇€瑕佸満鏅殑濮挎€佸瑙嗗浘鍥惧儚杩涜璁粌,RAF鏄涓€涓€氳繃閲囩敤鍩轰簬澶氬昂搴﹂潰鐗囩殑閴村埆鍣ㄥ紩鍏ヨ緪灏勫満鐢熸垚璁粌鐨勫鎶楁€ф鏋躲€傛渶杩戝凡缁忔姇鍏ヤ簡澶ч噺鐨勫姫鍔涙潵鏀硅繘鐢熸垚NeRF,渚嬪,GIRAFFE鐢ㄤ簬鍦ㄧ壒寰佺骇鍒紩鍏ヤ綋缁樺埗骞朵互鍙帶鐨勬柟寮忓垎绂诲璞″疄渚?#xff1b;Pi-GAN鐢ㄤ簬鍏锋湁SIREN鏋舵瀯鐨勫熀浜庤杽鑶滅殑璋冭妭鏂规;StyleNeRF鐢ㄤ簬闆嗘垚鍩轰簬鏍峰紡鐨勭敓鎴愬櫒,浠ュ疄鐜伴珮鍒嗚鲸鐜囧浘鍍忓悎鎴?#xff1b;EG3D鐢ㄤ簬鏈夋晥鐨勪笁骞抽潰琛ㄧず銆?/p>

鏈€杩?#xff0c;Jo绛変汉鎻愬嚭浜嗕竴绉?strong>鏉′欢鐢熸垚绁炵粡杈愬皠鍦?/strong>(CG NeRF),瀹冨彲浠ョ敓鎴愬弽鏄犻澶栬緭鍏ユ潯浠?#xff08;渚嬪鍥惧儚鎴栨枃鏈?#xff09;鐨勫瑙嗗浘鍥惧儚,濡傚浘鎵€绀恒€傚叿浣撹€岃█,閲囩敤棰勮缁冨壀杈戞ā鍨嬫彁鍙栨潯浠跺浘鍍忓拰鏂囨湰鐗瑰緛浣滀负NeRF鐨勮緭鍏ャ€傝櫧鐒剁敓鎴愮殑NERF鍙互淇濇寔瑙嗗浘涓€鑷存€?#xff0c;浣嗗畠浠敓鎴愮殑鍥惧儚涓嶅彲鏈湴缂栬緫銆備负浜嗗厠鏈嶈繖浜涢檺鍒?#xff0c;FENeRF鎻愬嚭浜嗕竴绉?D鎰熺煡鐢熸垚鍣?/strong>,鍙互鐢熸垚瑙嗗浘涓€鑷翠笖鍙湰鍦扮紪杈戠殑鑲栧儚鍥惧儚銆侳ENeRF浣跨敤涓や釜瑙h€︾殑娼滃湪浠g爜鍦ㄥ叿鏈夊叡浜嚑浣曠粨鏋勭殑绌洪棿瀵归綈鐨?D浣撶Н涓敓鎴愮浉搴旂殑闈㈤儴璇箟鍜岀汗鐞嗐€傚緱鐩婁簬杩欑搴曞眰3D琛ㄧず,Feneff鍙互鑱斿悎娓叉煋杈圭晫瀵归綈鐨勫浘鍍忓拰璇箟鎺╂ā,骞朵娇鐢ㄨ涔夋帺妯¢€氳繃GAN閫嗘槧灏勭紪杈?D浣撶Н銆?br />

基础入门教程

3.4.3銆佺敓鎴愬櫒閫嗘槧灏凬eRF

鏍规嵁3D鎰熺煡鍥惧儚鍚堟垚鐢熸垚绁炵粡缃戠粶鍑芥暟鐨勬渶鏂拌繘灞?#xff0c;涓€浜涘伐浣滄帰绱簡棰勮缁冪敓鎴愮缁忕綉缁滃洜瀛愮殑鍙嶆紨銆備即闅忕潃鏃ㄥ湪鏋勫缓鍩轰簬璇箟鎺╃爜鐨凬eRF鐨勬柊浠诲姟,Chen绛変汉閲囩敤浜嗗熀浜庣紪鐮佸櫒鐨勫弽杞?#xff0c;灏嗚涔夋槧灏勬槧灏勫埌棰勮缁冪殑pi-GAN鐨勬綔鍦ㄧ┖闂淬€?/p>

涓轰簡杩涗竴姝ユ彁楂橀€嗘槧灏勭殑鍑嗙‘鎬?#xff0c;Sem2NeRF灏?strong>鏂扮殑鍖哄煙鎰熺煡瀛︿範绛栫暐闆嗘垚鍒扮紪鐮佸櫒鍜岃В鐮佸櫒鐨勮璁′腑,骞朵娇鐢ㄦ彁鍙栫殑杞粨鍜岃窛绂诲満琛ㄧず鏉ュ寮鸿緭鍏ヨ涔夋帺鐮?IDE-3D閽堝鍏锋湁褰㈢姸鍜岀汗鐞嗗眬閮ㄦ帶鍒剁殑浜や簰寮?D鎰熺煡鍥惧儚缂栬緫,鎻愬嚭璁粌3D璇箟鎰熺煡鐢熸垚妯″瀷,璇ユā鍨嬪悓鏃剁敓鎴愯鍥句竴鑷寸殑浜鸿劯鍥惧儚鍜岃涔夋帺妯?#xff0c;鐒跺悗,浣跨敤涓や釜鍙嶅悜缂栫爜鍣ㄤ粠璇箟鍥剧敓鎴愭綔鍦ㄤ唬鐮佹槧灏?#xff0c;鏈€鍚庤鑼冪紪鐮佸櫒,浠ュ疄鐜拌鑼冭鍥句腑璇箟鎺╃爜鐨勬湁鏁堟搷浣溿€?/p>

涓庝笂杩板熀浜庣紪鐮佸櫒鐨勫弽婕斾笉鍚?#xff0c;CLIPNeRF閲囩敤鍩轰簬浼樺寲鐨勫弽婕?#xff0c;鏍规嵁鐭枃鏈彁绀哄疄鐜扮缁忚緪灏勫満鐨勬搷浣?/strong>銆傚湪StyleCLIP涓弿杩扮殑鍩轰簬鍓緫鐨勫尮閰嶆崯澶辩殑椹卞姩涓?#xff0c;鍓緫NeRF閫氳繃浣跨敤涓や釜浠g爜鏄犲皠鍣ㄦ潵浼樺寲娼滃湪浠g爜浠ュ疄鐜扮洰鏍囨搷绾?#xff0c;浠庤€屾ˉ鎺ョ敓鎴愭綔鍦ㄧ┖闂村拰鍓緫宓屽叆绌洪棿銆?/p>

鍙︿竴鏂归潰,鐢变簬鍖呮嫭浜嗙浉鏈哄Э鎬?#xff0c;鐢熸垚NeRF鐨勫弽杞粛鐒跺叿鏈夋寫鎴樻€с€備负浜嗗疄鐜扮ǔ瀹氱殑鏂囨湰寮曞鍥惧儚缂栬緫,StyleNeRF鎺㈢储浜?strong>缁撳悎鍩轰簬缂栫爜鍣ㄧ殑鍙嶆紨鍜屽熀浜庝紭鍖栫殑鍙嶆紨,鍏朵腑缂栫爜鍣ㄩ娴嬬浉鏈哄Э鎬佸拰閫氳繃閫嗕紭鍖栬繘涓€姝ョ粏鍖栫殑绮楁牱寮忎唬鐮併€?/p>

3.5銆佸叾浠栨柟娉?/h2>

闅忕潃鐢熸垚妯″瀷鍜岀缁忔覆鏌撶殑鍙戝睍,鍏朵粬鏂规硶涔熷疄鐜颁簡澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫(MISE),鍖呮嫭CLVA銆丆LIPstyler銆乀ext2Mesh銆丆LIPMes绛夌瓑,杩欓噷涓嶅啀灞曞紑,鍏蜂綋璇风湅鍘熸枃銆?/p>

3.6銆佹柟娉曟瘮杈?/h2>

鏂规硶 浼樼偣 缂虹偣
GAN 1銆侀珮淇濈湡鐨勫浘鍍忓悎鎴?#xff1b;2銆佸揩閫熺殑鎺ㄧ悊閫熷害;3銆侀珮FID鍜孖S鍒嗘暟; 1銆佽缁冧笉绋冲畾,瀹规槗妯″紡宕╁;2銆丆NN鏋舵瀯闅句互浠ョ粺涓€鏂瑰紡澶勭悊澶氭ā鎬佹暟鎹?/td>
鑷洖褰掓ā鍨?/td>

1銆佸钩绋虫€?#xff1b;2銆佹槗浜庢墿灞?#xff1b;3銆佸彲涓庢墿鏁fā鍨嬬粨鍚堜娇鐢?/td>

鎺ㄧ悊閫熷害鎱?/td>
鎵╂暎妯″瀷 1銆佸钩绋虫€?#xff1b;2銆佹槗浜庢墿灞?#xff1b;3銆佸缓妯℃墿鏁e厛楠屾椂浼樹簬鑷洖褰掓ā鍨?/td>

鎺ㄧ悊閫熷害鎱?/td>
鍩轰簬NeRF 1銆佸彲浠ュ緢濂藉鐞?D鍦烘櫙; 1銆侀檺鍒惰緝澶?#xff1b;2銆佽姹傛暟鎹泦鐨勫満鏅嚑浣曠畝鍗?#xff1b;3銆侀渶瑕佸叿鏈夋憚鍍忔満濮挎€佹敞閲婄殑澶氳鍥惧浘鍍?/td>

姝ゅ,鏈€鍏堣繘鐨勬柟娉曞€惧悜浜庣粨鍚堜笉鍚岀殑鐢熸垚妯″瀷浠ヤ骇鐢熶紭寮傜殑鎬ц兘銆備緥濡?#xff0c;Taming Transformer缁撳悎浜哣Q-GAN鍜岃嚜鍥炲綊妯″瀷浠ュ疄鐜伴珮鍒嗚鲸鐜囧浘鍍忓悎鎴?#xff0c;StyleNeRF缁撳悎浜哊eRF鍜孏AN浠ュ疄鐜伴珮淇濈湡搴﹀拰3D鎰熺煡鐨勫鎶楁€у浘鍍忓悎鎴?#xff1b;ImageBart灏嗚嚜鍥炲綊鍏紡涓庡椤瑰紡鎵╂暎杩囩▼鐩哥粨鍚?#xff0c;浠ュ悎骞朵粠绮楀埌缁嗙殑涓婁笅鏂囦俊鎭眰娆?#xff1b;X-LXMERT灏咷AN闆嗘垚鍒拌法妯℃€佽〃绀虹殑妗嗘灦涓?#xff0c;浠ュ疄鐜版枃鏈紩瀵肩殑鍥惧儚鐢熸垚銆?/p>

鍥涖€佸疄楠岄儴鍒?/h1>

浣滆€呮暣鐞嗕簡澶ч噺鐨勫疄楠?#xff0c;鍏蜂綋瀹為獙璇风湅鍘熸枃,杩欓噷浠呬粎鍒楀嚭閮ㄥ垎瀹為獙缁撴灉:


mysql 事务原理详解

浜斻€佸紑鏀惧紡鎸戞垬鍜屾湭鏉ユ帰绱?/h1>

灏界澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫杩戝勾鏉ュ彇寰椾簡鏄捐憲鐨勮繘灞?#xff0c;骞跺彇寰椾簡浼樺紓鐨勬€ц兘,浣嗕粛瀛樺湪涓€浜涙湭鏉ユ帰绱㈢殑鎸戞垬銆?/p>

6.1銆侀潰鍚戝ぇ瑙勬ā鐨勫妯℃€佹暟鎹泦

鐢变簬褰撳墠鏁版嵁闆嗕富瑕佷互鍗曚竴妯℃€?#xff08;渚嬪瑙嗚寮曞)鎻愪緵娉ㄩ噴,鐩墠澶у鏁板疄鐜癕ISE鐨勬ā鍨嬫槸鍗曚竴妯℃€佸舰寮?#xff0c;姣斿鏂囨湰鐢熸垚鍥惧儚,璇箟鍥惧儚鍚堟垚銆?/p>

鐒惰€?#xff0c;浜虹被鎷ユ湁鍦ㄥ绉嶆ā寮忕殑鎸囧涓嬪悓鏃跺垱寤鸿瑙夊唴瀹圭殑鑳藉姏銆備负浜嗘ā鎷熶汉绫绘櫤鑳?#xff0c;澶氭ā寮忚緭鍏ユ湁鏈涘湪鍥惧儚鐢熸垚涓悓鏃舵湁鏁堣瀺鍚堝拰鍒╃敤銆?/p>

6.2銆佹洿鍙潬鐨勮瘎浼版寚鏍?/h2>

澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫鐨勮瘎浼颁粛鐒舵槸涓€涓紑鏀剧殑闂銆傚埄鐢ㄩ璁粌妯″瀷杩涜璇勪及(濡侳ID)鏄棰勮缁冩暟鎹泦鐨勭害鏉?#xff0c;杩?strong>寰€寰€浼氶€犳垚涓庣洰鏍囨暟鎹泦鐨勫樊寮?/strong>銆傜敤鎴风爺绌?strong>鎷涘嫙浜虹被鍙楄瘯鑰呯洿鎺ヨ瘎浼板悎鎴愬浘鍍?#xff0c;浣嗚繖閫氬父鏄富瑙傜殑銆傚洜姝?#xff0c;璁捐鍑嗙‘鑰屽繝瀹炵殑璇勪及搴﹂噺瀵逛簬澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫鐨勫彂灞曢潪甯告湁鎰忎箟鍜屽叧閿€?/p>

鏈€杩?#xff0c;涓轰簡璇勪及鏂囨湰鍒板浘鍍忓悎鎴愪腑鐨勮法妯℃€佸榻?#xff0c;宸蹭娇鐢ㄩ璁粌CLIP鏉ユ祴閲忔枃鏈拰鐩稿簲鐢熸垚鍥惧儚涔嬮棿鐨勭浉浼煎害銆傚€煎緱娉ㄦ剰鐨勬槸,褰撳墠鍦ㄥ妯℃€佸涔犳柟闈㈢殑杩涙浣胯法妯℃€佷竴鑷存€ф洿涓虹簿纭?#xff0c;杩欏彲鑳芥湁鍔╀簬鏈潵寮€鍙戝彲闈犵殑璇勪及鎸囨爣銆?/p>

6.3銆侀珮鏁堢殑缃戠粶浣撶郴缁撴瀯

鐢变簬瀵瑰妯℃€佽緭鍏ョ殑鍥烘湁鏀寔鍜屽己澶х殑鐢熸垚寤烘ā,鍩轰簬鍙樻崲鍣ㄧ殑鑷洖褰掓ā鍨嬫垨鎵╂暎妯″瀷宸叉垚涓虹粺涓€澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫鐨勬柊鑼冨紡銆傜劧鑰?#xff0c;鑷洖褰掓ā鍨嬪拰鎵╂暎妯″瀷閮藉瓨鍦ㄦ帹鐞嗛€熷害鎱?/strong>鐨勯棶棰?#xff0c;杩欏湪楂樺垎杈ㄧ巼鍥惧儚鍚堟垚涓洿涓轰弗閲嶃€傛渶杩戠殑涓€浜涘伐浣滄帰绱簡鍔犻€熻嚜鍥炲綊妯″瀷鍜屾墿鏁fā鍨?#xff0c;鑰屽疄楠屼粎闄愪簬鍏锋湁浣庡垎杈ㄧ巼鐨勮瘯楠屾暟鎹泦,濡侰IFAR-10銆傚浣曞姞蹇嚜鍥炲綊妯″瀷鍜屾墿鏁fā鍨嬪湪瀹為檯搴旂敤涓殑鎺ㄧ悊閫熷害,浠嶇劧鏄湭鏉ユ帰绱㈢殑閲嶅ぇ鎸戞垬銆?/p>

鍙︿竴鏂归潰,鍩轰簬GAN鐨勬柟娉曞叿鏈夋洿濂界殑鎺ㄧ悊閫熷害,鑰屽箍娉涢噰鐢ㄧ殑CNN鏋舵瀯闃荤浜嗗畠瀵瑰妯℃€佽緭鍏ョ殑缁熶竴澶勭悊銆傛渶杩?#xff0c;涓€浜涘伐浣滄帰绱簡鍦℅ANs涓噰鐢ㄥ熀浜嶵ransformer鐨勬灦鏋?/strong>,杩欏彲鑳戒负寮€鍙戠敤浜庣粺涓€澶勭悊MISE浠诲姟鐨凣ANs鎻愪緵涓€浜涜瑙c€?/p>

6.4銆侀潰鍚?D

闅忕潃绁炵粡鍦烘櫙琛ㄧず妯″瀷鐗瑰埆鏄疦eRF鐨勫嚭鐜?#xff0c;3D鎰熺煡鍥惧儚鍚堟垚鍜岀紪杈戞湁鍙兘鎴愪负MISE鐨勪笅涓€涓獊鐮寸偣,鍥犱负瀹冩ā鎷熶簡鐪熷疄涓栫晫鐨?D鍑犱綍銆傞殢鐫€瀵规姉鎬ф崯澶辩殑鍔犲叆,鐢熸垚鎬eRF瀵筂ISE鍏锋湁鏄捐憲鐨勫惛寮曞姏,鍥犱负瀹冧笌娼滃湪绌洪棿鐩稿叧銆?/p>

褰撳墠鐨勭敓鎴愭€eRF妯″瀷(濡係tyleNeRF銆丒G3D)鑳藉浠庝竴缁勬湭瀹氫綅鐨?D鍥惧儚涓缓妯″叿鏈夌畝鍗曞嚑浣曚綋(濡備汉鑴搞€佹苯杞?#xff09;鐨勫満鏅?#xff0c;灏卞儚鏃犳潯浠禛AN(濡係tylgan)鐨勮缁冧竴鏍枫€?/p>

鍦ㄨ繖浜涘姫鍔涚殑鎺ㄥ姩涓?#xff0c;宸茬粡鎺㈢储浜嗕竴浜?D鎰熺煡MISE浠诲姟,渚嬪鏂囨湰鍒癗eRF鍜岃涔塼oNeRF銆傜劧鑰?#xff0c;褰撳墠鐢熸垚鐨凬ERF浠嶇劧鍦ㄥ叿鏈夊鏉傚嚑浣曞彉鍖栫殑鏁版嵁闆嗕笂鎸f墡,渚嬪DeepFashion鍜孖mageNet銆?br /> 浠呬粎渚濋潬瀵规姉鎬ф崯澶辨潵浠庢湭瀹氫綅鐨?D鍥惧儚涓涔犲鏉傜殑鍦烘櫙鍑犱綍缁撴瀯纭疄鏄洶闅惧拰鍏锋湁鎸戞垬鎬х殑銆備竴绉嶅彲鑳界殑瑙e喅鏂规鏄彁渚涘満鏅殑鏇村鍏堥獙鐭ヨ瘑,渚嬪,閫氳繃鐜版垚鐨勯噸寤烘ā鍨嬭幏寰楀厛楠屽満鏅嚑浣?#xff0c;涓虹敓鎴愭€т汉浣撳缓妯℃彁渚涢鏋跺厛楠屻€?/p>

鍊煎緱娉ㄦ剰鐨勬槸,鍦?D鎰熺煡椋庢牸杞崲鍜孨eRF娉涘寲鐨勪竴浜涜繎鏈熷伐浣滀腑,宸茬粡鎺㈢储浜嗗厛楠岀煡璇嗙殑鍔涢噺銆傚彟涓€绉嶅彲鑳界殑鏂规硶鏄彁渚涙洿澶氱殑鐩戠潱,渚嬪,鍒涘缓鍏锋湁澶氳鍥炬敞閲婃垨鍑犱綍淇℃伅鐨勫ぇ鍨嬫暟鎹泦銆備竴鏃?D鎰熺煡鐢熸垚寤烘ā鎴愬姛搴旂敤浜庡鏉傝嚜鐒跺満鏅?#xff0c;涓€浜涙湁瓒g殑澶氭ā鎬佸簲鐢ㄥ皢鎴愪负鍙兘,渚嬪DALL-E鐨?D鐗堟湰銆?/p>

鍏€佸簲鐢ㄤ笌褰卞搷

6.1銆佸簲鐢?/h2>

澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫鎶€鏈彲搴旂敤浜庤壓鏈垱浣滃拰鍐呭鐢熸垚,鍙?strong>骞挎硾鎯犲強璁捐甯堛€佹憚褰卞笀鍜屽唴瀹瑰垱浣滆€?/strong>銆傛澶?#xff0c;瀹冧滑鍙互鍦ㄦ棩甯稿簲鐢ㄤ腑琚皯涓诲寲,浣滀负娴佽濞变箰鐨勫浘鍍忕敓鎴愭垨缂栬緫宸ュ叿銆傛澶?#xff0c;浣滀负鍚堟垚鍜岀紪杈戠殑涓棿琛ㄧず鐨勫悇绉嶆潯浠跺ぇ澶х畝鍖栦簡鏂规硶鐨勪娇鐢?#xff0c;骞舵彁楂樹簡鐢ㄦ埛浜や簰鐨勭伒娲绘€с€傛€荤殑鏉ヨ,杩欎簺鎶€鏈?strong>澶уぇ闄嶄綆浜嗗叕浼楃殑闅滅,閲婃斁浜嗕粬浠湪鍐呭鐢熸垚鍜岀紪杈戞柟闈㈢殑鍒涢€犲姏銆?/p>

6.2銆佹互鐢?/h2>

鍙︿竴鏂归潰,涓嶆柇澧炲己鐨勭紪杈戣兘鍔涘拰鐢熸垚鐪熷疄鎰熶篃涓哄嚭浜庢伓鎰忕洰鐨勭敓鎴愭垨鎿嶇旱鍥惧儚鎻愪緵浜嗘満浼氥€傛互鐢ㄥ悎鎴愬拰缂栬緫鎶€鏈彲鑳戒紶鎾櫄鍋囨垨鎭舵剰淇℃伅,骞跺鑷磋礋闈㈢ぞ浼氬奖鍝嶃€?/p>

涓轰簡闃叉娼滃湪鐨勬互鐢?#xff0c;涓€绉嶆綔鍦ㄧ殑鏂规硶鏄?strong>寮€鍙戞娴嬬敤浜庤嚜鍔ㄨ瘑鍒獹AN鐢熸垚鎴栦吉鍥惧儚鐨勬妧鏈?/strong>,璇ユ妧鏈凡琚ぞ鍖虹Н鏋佺爺绌躲€傚悓鏃?#xff0c;鍦ㄩ儴缃插妯℃€佸浘鍍忓悎鎴愬拰缂栬緫鎶€鏈椂,搴?strong>浠旂粏鑰冭檻瓒冲鐨勬爡鏍忋€佹爣绛惧拰璁块棶鎺у埗,浠ュ敖閲忓噺灏戣鐢ㄩ闄┿€?/p>

6.3銆佺幆澧?/h2>

浣滀负鍩轰簬娣卞害瀛︿範鐨勬柟娉?#xff0c;褰撳墠鐨勫妯″瀷鐢熸垚鏂规硶涓嶅彲閬垮厤鍦?strong>闇€瑕丟PU鍜屽ぇ閲忕殑鑳介噺娑堣€楃敤浜庤缁冨拰鎺ㄧ悊,杩欏彲鑳藉湪澶ц妯′娇鐢ㄥ彲鍐嶇敓鑳芥簮涔嬪墠瀵圭幆澧冨拰鍏ㄧ悆姘斿€欎骇鐢熻礋闈㈠奖鍝嶃€?strong>寮卞寲瀵硅绠楄祫婧愰渶姹傜殑涓€涓柟鍚戝湪浜庣Н鏋佹帰绱㈡ā鍨嬫硾鍖栥€佽交閲忓寲銆備緥濡?#xff0c;鍦ㄥ悇绉嶆暟鎹泦涓帹骞跨殑棰勮缁冩ā鍨嬪彲浠ュぇ澶у姞蹇缁冭繃绋?#xff0c;鎴栦负涓嬫父浠诲姟鎻愪緵璇箟鐭ヨ瘑銆?/p>

鏈患杩版兜鐩栦簡澶氭ā鎬佸浘鍍忓悎鎴愬拰缂栬緫鐨勪富瑕佹柟娉曘€傚叾姒傝堪浜嗕笉鍚岀殑寮曞妯″紡,鍖呮嫭瑙嗚寮曞銆佹枃鏈紩瀵笺€侀煶棰戝紩瀵煎拰鍏朵粬妯″紡寮曞(濡傚満鏅浘)銆傜劧鍚庤缁嗕粙缁嶄簡涓昏鐨勫浘鍍忓悎鎴愬拰缂栬緫鑼冨紡:鍩轰簬GAN鐨勬柟娉曘€佽嚜鍥炲綊鏂规硶銆佸熀浜庢墿鏁g殑鏂规硶鍜屽熀浜嶯eRF鐨勬柟娉曘€傚叏闈㈣璁轰簡鐩稿簲鐨勪紭鍔垮拰鍔e娍,浠ユ縺鍙戝埄鐢ㄧ幇鏈夋鏋朵紭鍔跨殑鏂拌寖寮忋€傝繕瀵瑰熀浜庝笉鍚屾寚瀵兼ā寮忕殑MISE鏁版嵁闆嗗拰璇勪及鎸囨爣杩涜浜嗗叏闈㈣皟鏌ャ€傜劧鍚?#xff0c;鎴戜滑灏嗙幇鏈夋柟娉曞湪涓嶅悓MISE浠诲姟涓殑鎬ц兘鍒舵垚琛ㄦ牸骞惰繘琛屾瘮杈冦€傛渶鍚庢彁渚涗簡涓庢暣鍚堟墍鏈夋ā寮忋€佺患鍚堟暟鎹泦銆佽瘎浼版寚鏍囥€佹ā鍨嬫灦鏋勫拰3D鎰熺煡鐩稿叧鐨勫綋鍓嶆寫鎴樺拰鏈潵鏂瑰悜鐨勮鐐广€?/p>

璁烘枃鍦板潃:https://arxiv.org/abs/2112.13592v1
椤圭洰鍦板潃:https://github.com/fnzhan/MISE.

红黑树

鏈€鍚?/h1>

💖 涓汉绠€浠?#xff1a;浜哄伐鏅鸿兘棰嗗煙鐮旂┒鐢?#xff0c;鐩墠涓绘敾鏂囨湰鐢熸垚鍥惧儚(text to image)鏂瑰悜

stm32

📝 鍏虫敞鎴?#xff1a;涓澂鍙箰澶氬姞鍐?/a>

jdk动态代理

🔥 闄愭椂鍏嶈垂璁㈤槄:鏂囨湰鐢熸垚鍥惧儚T2I涓撴爮

WSA

🎉 鏀寔鎴?#xff1a;鐐硅禐👍+鏀惰棌猸愶笍+鐣欒█📝

ai

发表回复

您的电子邮箱地址不会被公开。 必填项已用*标注